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I.  BACKGROUND 


A.  RESEARCH  INTRODUCTION 

This  chapter  will  provide  an  overview  of  the  research  topic  starting  with  a 
background  on  how  the  Department  of  Defense  (DOD)  and,  more  specifically,  how  the 
Army  provides  materiel  solutions  for  their  Soldiers.  According  to  the  Office  of  the 
Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics,  and  Technology  (ASA 
[ALT]), 


the  Mission  of  the  Army  Acquisition  professional  field  is  to  provide 
our  Soldiers  a  decisive  advantage  in  any  mission  by  maintaining  quality 
acquisition  professionals  to  develop,  acquire,  field,  and  sustain  the 
world’s  best  equipment  and  services  through  efficient  leveraging  of 
technologies  and  capabilities  to  meet  current  and  future  Army  needs. 

The  reason  for  this  is  to  generate  and  prepare  the  best  equipped  Army 
that  maintains  the  technological  advantage  and  capabilities  against  any 
threat  in  any  environment.  (Assistant  Secretary  of  the  Army  for 
Acquisition,  Logistics,  and  Technology  (ASA  [ALT],  n.d.,  p.  1) 

The  Army  Acquisition  process  will  be  examined  in  this  paper  in  two  ways: 
through  the  lens  of  the  traditional  acquisition  process  utilizing  the  Joint  Capabilities 
Integration  Development  System  (JCIDS)  and  through  the  vantage  point  of  rapid 
equipping  through  joint  urgent  operational  needs  statements  (JUONS)  and  Operational 
Needs  Statements  (ONS).  In  addition,  the  Capabilities  Development  for  Rapid  Transition 
(CDRT)  process  will  be  examined  to  understand  better  how  the  Army  transitions  Rapid 
Acquisition  (RA)  programs  to  enduring  Programs  of  Records  (POR).  We  will  also 
address  the  Army  Materiel  Release  process  that  includes  several  different  mechanisms 
such  as  Full  Materiel  Release  (FMR)  and  the  Urgent  Materiel  Release  (UMR). 

After  describing  the  acquisition  avenues  available  to  the  Army  the  purpose, 
importance,  and  criticality  of  Test  and  Evaluation  (T&E)  activities  will  be  explored.  This 
background  information  will  establish  a  relevant  baseline  that  can  be  used  throughout  our 
research  to  identify  potential  risk  and  or  short  comings  associated  with  RA  as  they  move 
to  enduring  capabilities.  We  used  two  specific  research  questions  to  analyze  the  data  and 
apply  it  to  our  problem  statement: 
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1.  Did  the  abbreviated  T&E  requirements  in  an  RA  environment  dilute  the 
important  role  of  the  “honest  broker”  that  a  T&E  team  should  play  in  an 
acquisition  program? 

2.  What  are  the  long-term  effects  on  programs  from  the  manner  that  T&E  is 
conducted  to  meet  RI  timelines? 

This  report  will  address  our  thesis  topic  concerning  T&E  in  an  RA  environment. 
Our  research  protocols  and  research  rational  will  also  be  provided.  The  following 
sections  provide  background  acquisition  information  for  requirements  generation,  the  role 
of  T&E,  and  the  NS-E  transition  process. 

1.  Requirements  Generation 

The  DOD  is  dependent  on  equipment  and  technology.  As  technology  develops 
faster  the  DOD  has  developed  a  system  to  determine  which  technology  is  used.  The 
JCIDS  process  is  used  to  assist  the  DOD  in  requirements  generation. 

JCIDS  is  a  need  driven  joint  capabilities-based  requirements  generation 
process.  The  objective  is  to  develop  a  balanced  and  synchronized 
Doctrine,  Organization,  Training,  Materiel,  Leadership,  Personnel, 
Facilities,  and  Policy  (DOTMLPF-P)  solution  approach  that  is 
affordable,  militarily  useful,  supportable  by  outside  agencies,  and  based 
on  mature  technology  that  is  demonstrated  in  a  relevant  operational  or 
laboratory  environment.  (Army  Force  Management  School,  2013,  p.  6) 

The  JCIDS  process  uses  a  deliberate,  structured,  and  thorough  process  to  generate 
requirements  (illustrated  in  the  first  portion  of  Figure  1,  leading  to  the  full  acquisition 
process)  and  can  be  viewed  as  the  kickoff  of  a  new  Army  acquisition  program.  Several 
years  may  be  spent  on  up  front  analysis  to  ensure  DOTMLPF-P  considerations  are 
adequately  addressed  and  appropriate  steps  are  taken  during  T&E  planning  and 
execution. 

JUONS  are  created  by  Combatant  Commanders  (CCDR)  in  order  to  identify  a 
multi-service  capability  shortfall  for  a  force  preparing  to  deploy  or  those  currently 
deployed.  This  process  is  not  designed  to  circumvent  the  JCIDS  process  but  used  in  a 
limited  manner  to  provide  a  rapid  capability  for  a  very  specific  purpose.  Field 
Commanders  identify  capability  gaps  and  forward  those  gaps  to  the  CCDR’s  staff. 
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Figure  1 .  JCIDS  Process  Overview 
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From  R.  James,  personal  communication,  January  9,  2015. 


The  staff  then  determines  whether  the  requirement  meets  the  threshold  of  a 
JUONS.  If  all  the  requirements  for  a  JUONS  are  met,  the  CCDR  submits  the  JUONS 
formally  to  the  Joint  Staff  for  action.  The  Joint  Force  Structure,  Resources,  and 
Assessment  Directorate  (Joint  Staff  J8)  either  validates  or  rejects  the  JUONS.  If 
validated,  the  JUONS  is  sent  to  the  Joint  Rapid  Acquisition  Cell  (JRAC)  for  resourcing 
and  action.  The  JRAC  assigns  the  action  to  an  organization  to  fulfill  the  need  and  sustain 
it  for  up  to  24  months  following  fielding. 

The  Army  ONS  process  is  used  when  operational  commanders,  in  the  grade  of  06 
and  higher,  identify  a  capability  gap.  The  commander’s  unit  must  be  deployed,  preparing 
to  deploy  or  a  part  of  a  high  priority  unit.  The  unit  produces  an  ONS  and  begins  the 
process  by  submitting  it  through  the  Equipment  Common  Operating  Picture  (ECOP) 
system.  The  request  then  moves  through  the  approval  process.  If  the  unit  is  deployed,  the 
request  is  routed  through  deployed  unit’s  chain  of  command  to  the  Army  Service 
Component  Commander  (ASCC). 

Non-deployed  units  submit  requests  through  the  major  command  to  which  they 
are  assigned.  Once  the  ONS  is  approved  by  the  respective  ASCC,  it  moves  on  to 
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Headquarters  Department  of  the  Army  (HQDA)  Deputy  Chief  of  Staff  for  Operations 
(G3)  for  validation.  If  validated,  the  ONS  moves  to  HQDA  Deputy  Chief  of  Staff  for 
Resource  Management  (G8)  for  resourcing.  Resourcing  is  the  process  of  providing 
budget  authority  to  a  PM  for  execution.  If  resourcing  is  available,  the  validated  request  is 
forwarded  with  funding  to  the  acquisition  community  for  action.  A  program  manager  is 
assigned  the  requirement  and  allocated  resources  to  carry  out  with  an  acquisition  plan. 

2.  Role  of  Test  and  Evaluation 

Test  and  Evaluation  plays  a  key  role  in  acquisition  programs.  “Testing  is  the 
process  of  obtaining,  verifying,  or  providing  data  to  determine  whether  an  item  meets  or 
fails  to  meet  defined  objectives.  Evaluation  is  the  analysis  of  data  to  assess  progress  of 
design,  performance,  supportability,  or  other  required  attributes”  (W.  Chadwick,  personal 
communication,  January  16,  2014).  The  Defense  Acquisition  Guidebook  (DAG)  states 
that 


a  rigorous  and  efficient  T&E  program  provides  early  knowledge  of 
developmental  and  operational  issues.  Correcting  these  issues  early 
enough  can  mitigate  risks  of  cost  overruns  and  schedule  slippages,  and 
can  ultimately  contribute  to  delivery  of  effective  and  suitable  weapons, 
Information  Technology  (IT)  and  National  Security  Systems  (NSS)  to 
the  Warfighters  in  a  timely  manner.  (Defense  Acquisition  University, 

2015a,  p.773) 

The  Army  Test  and  Evaluation  Command  (ATEC)  strives  “to  be  a  team  of  highly 
skilled  test  and  evaluation  professionals  focused  on  informing  equipping  decisions  for 
today’s  and  tomorrow’s  warfighter”  (ATEC,  n.d.).  Army  policy  assigns  ATEC  as  the 
responsible  organization  for  Operational  Utility  Assessments. 

Under  an  RA  environment  with  an  abbreviated  fielding  schedule,  the  systems 
engineers,  test  leads,  the  product  managers,  and  the  entire  Integrated  Product  Team  (IPT) 
needs  to  ensure  that  reduced  T&E  events  still  provide  the  appropriate  level  of  data, 
analysis,  and  technical  confidence  that  the  product  satisfies  user  needs. 


4 


3. 


Materiel  Release 


The  Army  uses  the  materiel  release  process  to  ensure  that  equipment  is  safe, 
suitable,  and  logistically  supportable.  Army  Regulation  (AR)  700-142  (Type 
Classification,  Materiel  Release,  Fielding,  and  Transfer)  establishes  policy  and 
prescribes  procedures  for  the  Army’s  Type  Classification  (TC),  Materiel  Release  (MR), 
material  fielding,  and  material  transfer  processes. 

There  are  four  types  of  materiel  release:  Full  Materiel  Release  (FMR),  Conditional 
Materiel  Release  (CMR),  Urgent  Materiel  Release  (UMR),  and  training  materiel  release 
(TMR).  The  definitions  and  requirements  for  each  are  found  in  the  Appendix. 

A  FMR  allows  Program  Managers  (PM)  to  field  non-developmental  items  or 
commercial  items  to  soldiers.  In  a  developmental  effort,  an  FMR  allows  the  PM  to  move 
on  to  a  Full  Rate  Production  (FRP)  decision.  The  three  main  characteristics  being 
evaluated  are  safety  and  hazards,  suitability,  and  supportability.  There  are  32  overall 
activities  that  can  be  evaluated.  While  each  of  the  32  activities  or  documents  require 
some  level  of  testing  or  analysis,  T&E  plays  the  largest  role  in  the  evaluation  of  activities 
17  (ATEC  MR  position  memorandum)  and  18  (ATEC  OTA  Milestone  Assessment 
Report  (OMAR)  or  OTA  Evaluation  Report  (OER)).  ATEC  is  responsible  for  carrying 
out  T&E  efforts  according  to  the  Test  &  Evaluation  Master  Plan  (TEMP)  in  order  to 
show  how  the  system  meets  the  established  requirements  identified  in  the  capabilities 
documents.  When  the  PM  can  demonstrate  through  T&E  that  the  requirements  for  an 
FMR  are  met,  the  MDA  will  grant  the  FMR. 

A  CMR  is  used  for  various  reasons  when  the  conditions  for  a  FMR  cannot  be  met. 
A  CMR  is  appropriate  when  FRP  is  not  a  required  part  of  the  program.  When  the  Army 
Acquisition  Executive  (AAE)  allows  the  program  to  proceed  on  to  FRP  with  only  a 
CMR,  a  PM  is  prepared  to  execute  Low  Rate  Initial  Production  (LRIP)  and  will  meet  all 
the  requirements  of  the  FMR  prior  to  FRP.  If  an  upgrade  is  scheduled  post  FRP  that  will 
meet  the  all  requirements. 

A  UMR  is  used  in  very  specific  situations  in  which  the  Army  requires  a  capability 
to  be  fielded  rapidly  to  a  unit  that  has  an  operational  need  and  is  deployed  or  preparing  to 
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deploy  for  an  approved  contingency.  An  UMR  is  generated  in  response  to  an  ONS  or  a 
Department  of  the  Army  Deputy  Chief  of  Staff  for  Operations,  Plans  and  Training 
(HQDA  G3/5/7)  Directed  Requirement.  HQDA  G3/5/7  is  the  Army  lead  for  determining 
what  capabilities  the  Army  is  required  to  have.  HQDA  G3/5/7  is  located  in  the  Pentagon 
and  led  by  a  Lieutenant  General.  An  UMR  is  limited  to  the  specific  quantity,  location, 
and  application  as  laid  out  by  the  approved  requirement.  The  Commanding  General  (CG) 
of  the  gaining  organization  will  retain  all  equipment  until  operational  contingency 
requirements  are  met.  The  main  focus  of  UMR  documentation  requirements  is  to  ensure 
the  safety  of  the  users,  validate  the  requirements  (e.g.,  a  JUONS  or  HQDA  directed 
requirement),  and  document  and  notify  the  CG  (or  other  requestor)  of  all  known 
equipment,  supportability,  and  sustainment  issues  with  the  CG’s  acceptance  statement. 
Since  the  UMR  is  for  a  specific  capability  demonstrated  in  some  capacity  for  a  specific 
location,  many  of  the  documentation  and  activities  associated  with  a  FRP  decision  are  not 
required  here.  For  example,  a  completed  TEMP  (and  associated  activities)  is  not  a 
prerequisite  for  a  successful  UMR. 

A  TMR  is  used  by  a  PM  when  materiel  solutions  are  to  be  used  by  the  Army 
Training  and  Doctrine  Command  (TRADOC)  to  develop  periods  of  instructions  and 
curriculum.  Items  released  under  a  TMR  may  be  a  prototype,  or  produced  in  a  manner 
different  from  normal  production  or  may  have  not  met  the  requirements  for  a  FMR. 

4.  Non-Standard  Equipment  Transition  Process 

The  Army  has  transitioned  rapidly  developed  capabilities  into  enduring 
requirements  once  proven  valuable  in  operational  theaters.  Using  Army  Directive  2010- 
07  to  identify  which  valuable  capabilities  to  maintain  and  integrate  into  the  Army 
capability  baseline,  the  HQDA  G3/5/7  DAMO-CI  and  TRADOC  developed  the 
Capabilities  Development  for  Rapid  Transition  (CDRT)  process  (Army  Training  and 
Doctrine  Command  [TRADOC],  2012,  p.  1). 

The  purpose  of  the  Army  CDRT  process  is  to  identify  valuable  capabilities  that 
have  been  developed  to  meet  emerging  challenges.  “The  process  identifies,  through 
operational  Army  unit  input,  systems  working  well  in  operational  theaters  and  speeds  the 
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process  to  get  them  into  the  hands  of  Soldiers  throughout  the  Army  for  the  long  term,  and 
reduces  expenditures  by  terminating  sustainment  funding  of  systems  no  longer  required” 
(TRADOC,  2012,  p.  2).  The  HQDA  G3/5/7  DAMO-CI  and  TRADOC,  quarterly  identify 
the  best  nominated  non-standard  materiel  and  non-materiel  insertions  the  Army  should 
incorporate  throughout  the  force:  “The  goal  is  to  significantly  reduce  the  time  needed  to 
field  selected  systems  or  capabilities  to  the  operational  Army”  (TRADOC,  2012,  p.  1). 
Tactical  NS-E  transitioning  to  Acquisition  Program  Candidates  (APCs)  will  be  sustained 
in  theater  using  Overseas  Contingency  Operations  (OCO)  funding  until  approved  JCIDS 
documentation  and  Program  Objective  Memorandum  (POM)  funding  is  in  place 
(TRADOC,  2012,  p.  1). 

The  CDRT  process  was  developed  to  bring  programs  and  systems  developed  and 
fielded  quickly  under  a  steadier  funding  profile  for  long-term  sustainment.  Gaps  that 
were  not  addressed  during  the  initial  development  and  fielding  would  be  addressed  once 
the  system  secured  POM  funding. 

A  capability  considered  under  CDRT  can  go  in  one  of  three  directions:  selection 
as  APC,  sustainment  of  the  capability,  or  terminate  the  capability.  A  capability  selected 
as  an  APC  has  shown  to  fill  a  current  operational  need,  is  theater-proven,  and  is 
applicable  to  the  entire  Army  and  future  force  structures.  The  program  is  intended  to 
enter  the  JCIDS  process  at  Milestone  B  (MS-B)  or  Milestone  C  (MS-C),  or  merge  into 
existing  programs.  The  program  will  begin  to  submit  for  program  funding  in  the  POM 
but  sustain  current  funding  with  a  bridge  resourcing  strategy  through  OCO  funding  (R. 
Mason,  personal  communication,  unpublished  PowerPoint,  December  14,  2012).  A 
capability  selected  as  ‘sustain’  fills  current  theater  operational  need  but  without  broader, 
enduring  applications  to  the  entire  Army  or  future  force  structure.  It  is  recognized  as 
equipment  for  specific  theater  use  only  and  will  be  sustained  for  that  theater  (R.  Mason, 
personal  communication,  unpublished  PowerPoint,  December  14,  2012).  A  capability 
selected  for  termination  does  not  fulfill  its  intended  function  or  performs  unacceptably. 
The  rationale  can  include  obsolescence,  better  alternatives,  or  an  already  active 
replacement  activity.  The  capability  can  be  sustained  in  theater  only  if  retained  and 
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funded  by  a  specific  unit  (R.  Mason,  personal  communication,  unpublished  PowerPoint, 
December  14,  2012). 

Figure  2  outlines  the  process  a  capability  must  be  validated  through  in  order  to  be 
considered  an  APC  and  enter  into  the  JCIDS  process.  To  be  eligible  for  CDRT,  a  capability 
is  required  to  be  operationally  mature,  in  theater  for  a  minimum  of  120  days,  and  have  a 
completed  a  Forward  Operational  Assessment  (FOA).  A  FOA  is  conducted  by  ATEC,  in  the 
theater  of  operations  where  RA  are  deployed  to  determine  the  suitability  of  the  capability. 
The  capability  can  be  nominated  for  consideration  by  various  sources  or  organizations.  The 
candidate  list  is  then  assessed  and  prioritized  with  lead  capabilities  identified  (R.  Mason, 
personal  communication,  unpublished  PowerPoint,  December  14,  2012). 


Figure  2.  RA  to  POR  Process  Flow 
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From  R.  Mason,  personal  communication,  unpublished  PowerPoint,  December  14,  2012. 


Then  the  capabilities  are  recommended  for  termination,  sustain,  or  APC  status 
The  decision  is  validated  through  and  reviewed  by  a  Council  of  Colonels,  the  Director  of 
the  Army  Capabilities  Integration  Center  (ARCIC),  HQDA,  and  TRADOC  to  ensure  the 
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capability  meets  DOTMLPF-P  considerations  for  current  and  future  Warfighter  needs. 
Finally,  the  Army  Requirements  Oversight  Council  (AROC)  approves  or  disapproves  the 
recommendations.  The  approved  capabilities  move  into  the  JCIDS  process  (with  HQDA 
processing  the  documentation),  bypassing  the  traditional  capabilities  based  analysis  phase 
with  either  a  Capability  Production  Document  (CPD)  or  Capability  Development 
Document  (CDD)  (TRADOC,  2012,  p.  1). 

Once  the  AROC  approves  the  CDRT  recommendations,  HQDA  (through 
TRADOC  headquarters)  tasks  a  TRADOC  center  or  other  combat  developers  to  produce 
the  required  JCIDS  documentation.  HQDA  is  critical  in  ensuring  funding  is  aligned  with 
required  capabilities  across  the  POM.  A  system  is  considered  an  acquisition  program 
once  it  has  an  approved  CPD,  a  MS-C  decision,  and  funding  in  the  base  budget. 
Capabilities  transitioning  to  APCs  will  be  sustained  in  theater  using  OCO  funding  until 
approved  JCIDS  documentation  and  POM  funding  is  in  place  (TRADOC,  2012,  p.  1). 

As  of  2012,  twelve  iterations  of  the  CDRT  process  have  been  conducted  with  the 
thirteenth  being  processed.  Through  Iteration  13,  the  CDRT  process  has  considered  419 
materiel  systems  and  13  non-materiel  capabilities  (some  more  than  once).  The  AROC  has 
approved  32  systems  for  acquisition  program  status  and  ten  non-materiel  capabilities  as 
enduring  (TRADOC,  2012,  p.  2). 

A  similar  process  to  CDRT  was  named  the  Non-Standard  Equipment  (NS-E) 

Army  Requirements  Oversight  Council  (AROC)  process.  While  CDRT  focused  on 

identifying  systems  as  candidates  to  transition  into  the  JCIDS  process  with  a  desired  end 

state  as  a  POR,  the  NS-E  AROC  process  focused  on  identifying  continued  funds  to 

maintain  a  capability  portfolio  (but  not  necessarily  as  a  POR).  The  review  takes  into 

consideration  the  costs  to  retain  a  system,  how  the  system  fits  into  the  Army  equipping 

strategy,  the  cost  to  buy  new  (either  exact  or  a  very  similar  capability),  and  best  of  breed 

considerations  (R.  Mason,  personal  communication,  unpublished  PowerPoint,  December 

14,  2012).  NS-E  AROC  recommendations  have  four  options  (Retain,  Invest,  Maintain,  or 

Divest)  and  implementation  in  a  HQDA  Memorandum  approved  by  the  Vice  Chief  of 

Staff  of  the  Army  (VCSA).  Retained  systems  are  identified  with  known  quantities  and 

will  have  a  resourcing  requirement  established  for  the  POM.  An  Invest  recommendation 
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allows  for  further  development  of  the  capability  (in  the  form  of  further  RDT&E  or 
science  and  technology  (S&T)  work)  without  retaining  the  equipment  itself.  The 
capability  could  also  be  transferred  to  another  program  of  record.  If  a  Maintain 
recommendation  is  given,  the  Army  retains  the  equipment  for  operations  but  will  then 
divest  of  it  once  available  current  funding  ends,  the  equipment  is  no  longer  needed  for 
operations,  or  there  is  a  cessation  of  hostilities.  A  Divest  recommendation  means  that 
equipment  must  be  removed  from  the  inventory  even  for  the  current  fight  (R.  Mason, 
personal  communication,  unpublished  PowerPoint,  December  14,  2012). 

5.  Summary 

This  section  provided  a  background  on  how  the  DOD  addresses  requirements 
using  the  JCIDS  process,  how  the  DOD  and  the  Army  addresses  urgent  requirements 
using  JUONS  and  ONS,  and  why  the  Army  conducts  T&E,  and  finally  how  the  Army 
achieves  a  MR  for  new  materiel  solutions.  It  is  the  responsibility  of  the  generating  force 
to  ensure  the  operating  force  is  equipped  in  a  safe,  suitable  and  effective  manner.  The 
acquisition  community  uses  the  JCIDS,  JUONS/ONS,  and  the  MR  process  to  guide  the 
development  of  materiel  solutions.  In  some  cases  these  processes  overlap  and  in  other 
cases  large  gaps  exists  that  can  create  risk  for  Warfighters.  This  project  will  analyze  how 
the  operating  force  is  equipped  using  each  process  as  the  capability  moves  from  a  rapid  to 
an  enduring  requirement. 

B.  PROBLEM  STATEMENT 

The  Department  of  Defense  (and  more  specifically  the  Army)  has  compressed  the 
acquisition  process  in  order  to  field  capability  faster  to  units  deployed  in  conflict.  This 
compression  has  reduced  the  amount  of  T&E  that  is  conducted  up  front  and  may  not  be 
fully  addressed  in  future  POR. 

Thirteen  years  of  combat  operations,  numerous  other  contingency,  humanitarian 

aid,  and  civil  support  operations  has  generated  the  need  for  capability  to  be  delivered 

faster  than  the  normal  acquisition  process  can  accommodate.  While  the  JCIDS  process 

was  developed  to  produce  materiel  solutions  for  the  services,  based  on  known  capability 

gaps,  it  is  a  very  deliberate  process  that  requires  a  significant  investment  in  time  and 
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resources  that  may  not  be  available.  The  services  have  developed  processes  to  deliver 
capabilities  faster  than  the  traditional  acquisition  process.  This  has  created  potential  gaps 
and  risks  associated  with  delivering  rapid  capabilities  due  to  the  unforeseen  length  of  a 
system  life  cycle  that  may  not  have  been  taken  into  account  during  initial  T&E  activities. 
Requirements  for  RA  often  are  not  written  with  enough  fidelity  to  inform  the  T&E 
community. 

C.  RESEARCH  OBJECTIVES 

The  objective  of  this  research  is  to  determine  the  differences  in  T&E  between  RA 
and  POR.  Additional  attention  will  be  given  to  the  tradeoffs  specific  programs,  such  as 
the  Counter  Radio  Controlled  Improvised  Explosive  Device  (RCIED)  Electronic  Warfare 
(CREW)  and  the  Dismounted  Reconnaissance  Sets  Kits  and  Outfits  (DR  SKO),  made  in 
terms  of  cost,  performance,  and  schedule. 

D.  THESIS  STATEMENT 

This  study  will  explore  the  effect  that  the  RA  environment  has  had  on  the  Army 
acquisition  process  during  the  recent  conflicts  in  Iraq  (Operation  Iraqi  Freedom  and 
Operation  New  Dawn)  and  in  Afghanistan  (Operation  Enduring  Freedom).  In  order  to 
achieve  extremely  quick  fielding  timelines,  the  Army  often  utilized  the  UMR  fielding 
process,  which  reduces  the  amount  of  required  T&E  events.  The  analysis  is  designed  to 
assess  the  benefits  or  drawbacks  of  using  the  UMR  process  (with  limited  T&E  activities) 
for  Army  acquisition  programs.  Recommendations  will  be  made  for  PMs  that  field  RA  on 
how  to  best  capitalize  on  previous  T&E  activity  that  can  be  used  as  the  RA  transitions  to 
a  POR  while  still  realizing  rapidly  fielding  equipment. 

E.  RESEARCH  QUESTIONS 

The  following  research  questions  were  used  to  guide  our  data  gathering  and 
associated  analysis. 

Research  Question  1:  Did  the  abbreviated  T&E  requirements  in  a  RA 
environment  dilute  the  important  role  of  the  “honest  broker”  that  a  T&E  team  should  play 
in  an  acquisition  program? 
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Research  Question  2:  What  are  the  long-term  effects  on  programs  from  the 
manner  that  T&E  is  conducted  to  meet  RA  timelines? 

F.  PURPOSE/BENEFIT 

While  RA  programs  were  able  to  address  urgent  needs  quickly  during  asymmetric 
warfare,  they  may  end  up  being  victims  of  their  own  success.  This  study  will  produce 
“lessons  learned”  from  abbreviated  T&E  activities.  Program  Offices  can  use  this 
information  to  mitigate  risk  in  future  research  efforts  or  rapid  fielding  initiatives.  Army 
organizations  can  also  benefit  by  implementing  small  changes  to  their  development 
methodologies  that  will  improve  the  long-term  posture  and  sustainability  of  new 
capabilities. 

G.  REPORT  ORGANIZATION 

The  remainder  of  this  report  is  organized  as  follows:  Chapter  II  will  focus  on 
detailed  literary  examination  of  each  research  question.  Chapter  III  explores  the 
foundational  and  program  specific  data  related  to  the  research.  Chapter  IV  will  present 
our  findings  with  our  analytical  approach  and  resultant  findings.  Chapter  V  will  close 
with  our  conclusions  and  recommendations.  There  are  three  appendices  with  backup 
information. 
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II.  LITERATURE  REVIEW 


A.  FOUNDATIONAL  BACKGROUND  LITERATURE 

Research  for  this  project  began  with  an  intense  review  of  relevant  literature.  In 
order  to  form  a  complete  understanding  of  how  the  DOD  fills  capability  gaps  and 
confirms  solutions,  information  was  gathered  from  a  variety  of  sources  that  included 
DOD  Instructions,  Defense  guidebooks,  Army  regulations,  Army  posture  statements,  and 
pamphlets. 

Any  research  involving  acquisition  requires  the  study  and  understanding  of  both 
Department  of  Defense  Instruction  (DODI)  5000.1  and  DODI  5000.2.  DODI  5000.1  lays 
out  the  overarching  policy  that  all  DOD  organizations  must  follow  in  order  to  manage 
acquisition  programs.  It  explains  what  the  Defense  Acquisition  System  should  do  in  order 
to  provide  capabilities  required  by  the  National  Security  Strategy.  DODI  5000.2  takes 
this  guidance  and  provides  the  framework  for  how  PM’s  use  the  Defense  Acquisition 
System  to  manage  programs  and  provide  necessary  capabilities  to  the  warfighter. 

There  were  two  guidebooks  that  required  study:  The  Defense  Acquisition 
Guidebook  and  The  Test  and  Evaluation  Management  Guide. 

By  design,  The  Defense  Acquisition  Guidebook  (DAG)  supports  both  DODI 
5000.1  and  5000.2.  It  is  commonly  referred  to  as  a  desk  reference  for  the  acquisition 
workforce.  The  majority  of  the  research  for  this  document  came  from  DAG  Chapter  9, 
“Test  and  Evaluation.”  This  chapter  documents  the  key  ingredients  necessary  to  build  a 
T&E  program  that  is  effective  and  meets  the  requirements  of  DODI  5000.2.  Chapter  4 
was  also  used  in  order  to  reference  the  Systems  Engineering  process  (Department  of 
Defense,  2015). 

The  Test  and  Evaluation  Management  Guide  is  another  desk  reference  that 
supports  the  Defense  Acquisition  University  (DAU)  efforts  to  educate  the  Acquisition 
Workforce.  It  is  used  to  facilitate  T&E  course  work,  and  in  the  field,  to  provide  a  better 
understanding  of  the  T&E  process.  While  the  guidebook  covered  all  aspects  of  T&E,  our 
researched  delved  into  Chapters  5,  6,  and  10.  These  chapters  focused  on  the  T&E 
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process,  Development  Test  and  Evaluation  (DT&E),  and  Operational  Test  and  Evaluation 
(OT&E)  respectively  (Defense  Acquisition  University,  2015a). 

There  were  three  distinct  Army  Regulations  (AR)  that  were  studied  as  part  of  this 
research  effort:  AR  700-142,  AR  71-9  and  AR  385-10. 

AR  700-142  ( Type  Classification,  Materiel  Release,  Fielding,  and  Transfer ) 
identifies  the  policies  and  procedures  for  the  acceptance  of  materiel  solutions  by  the 
Army.  This  regulation  is  the  safeguard  that  ensures  materiel  solutions  are  ready  for  each 
stage  of  the  acquisition  process.  Chapter  4  “Materiel  Release”  covers  the  required  actions 
a  PM  must  complete  in  order  to  receive  a  MR  from  the  Milestone  Decision  Authority 
(Department  of  the  Army,  2008). 

AR  71-9  Warfighting  Capabilities  Determination,  establishes  how  the  Army 
implements  the  JCIDS  process.  Chapter  3  specifically  lays  out  the  responsibility  of  the 
AROC  and  how  requirements  are  generated  by  the  Army  (Department  of  the  Army, 
2009d).  To  further  explore  how  RA  capabilities  become  enduring  capabilities,  the  NS-E 
and  CDRT  processes  were  explored  as  well. 

A  key  piece  to  any  materiel  solution  is  the  safety  aspect  of  the  system.  AR  385— 
10,  The  Army  Safety  Program,  is  the  cornerstone  document  for  safety  in  the  Army 
(Department  of  the  Army,  2013b).  This  regulation  is  key  to  the  management  of  safety 
risks  that  are  identified  for  Army  programs  as  well  as  any  applicable  civilian  statutory 
requirements.  Additional  research  to  support  AR  385-10  was  conducted  using 
Department  of  the  Army  Pamphlet  (DA  PAM)  385-16,  System  Safety  Management 
Guide  (Department  of  the  Army,  2013a).  This  document  is  extremely  important  to 
Program  Executive  Offices  (PEO)  and  PMs  because  it  shows  how  the  acquisition 
community  should  manage  safety  programs.  In  particular,  Chapter  4  “System  Safety  for 
Testers  and  Evaluators”  lays  out  the  methodology  for  PM’s  and  testers  to  uncover  safety 
problems  in  order  to  manage  or  eliminate  them. 

Probably  the  single  most  important  document  researched  for  this  project  was 
ATEC  Regulation  73-1  ( System  Test  and  Evaluation  Policy).  This  documents  how  ATEC 
supports  the  acquisition  process  through  T&E  activities.  More  importantly,  it  provides 
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valuable  insight  into  how  the  test  community  thinks  and  the  rationale  behind  their 
decisions.  Much  of  the  information  found  in  this  PAM  cannot  be  found  elsewhere  in 
literature  (Army  Test  and  Evaluation  Command,  2013). 

Additionally,  the  2012  TRADOC  Army  Posture  Statement  on  CDRT  and  Army 
Directive  2010-07  were  referenced  to  understand  the  processes  available  for  a  RA  to 
transition  to  a  POR. 

B.  PROGRAM  DATA  LITERATURE 

Once  our  basic  research  was  complete,  we  decided  to  examine  real  world 
examples  that  moved  from  a  JUONS/ONS  to  a  POR.  For  this  research,  we  focused  on  the 
Dismounted  Reconnaissance  Sets  Kits  and  Outfits  (DR  SKO)  and  the  Counter  Remote 
Control  Improvised  Explosive  Device  Electronic  Warfare  (CREW)  POR. 

The  first  set  of  documents  reviewed  was  the  TEMP.  The  TEMP  lays  out  what  will 
be  tested  and  how  testing  will  occur  across  the  life  cycle  of  the  program.  It  includes  all 
required  tests  in  order  for  the  PM  to  meet  the  goals  established  in  the  Acquisition 
Strategy.  The  TEMP  is  required  and  approved  at  the  MS-B  (Defense  Acquisition 
University,  2015b). 

The  second  set  of  documents  reviewed  was  the  Systems  Engineering  Plan  (SEP). 
The  SEP  much  like  the  TEMP  is  approved  at  MS-B  and  is  a  key  document  to  the 
execution  of  a  program.  The  SEP  details  how  the  technical  aspects  of  a  program  will 
be  executed.  The  SEP  is  used  as  a  reference  that  is  continually  updated  throughout 
the  product  life  cycle,  making  adjustments  along  the  way  (Defense  Acquisition 
University,  2015b). 
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III.  DATA 


A.  FOUNDATIONAL  DATA 

The  data  presented  in  this  section  provides  the  reader  with  the  required  working 
knowledge  to  understand  the  T&E  requirements  and  activities  within  POR  and  RA. 

1.  T&E  Requirements  for  POR 

To  establish  a  control  measure  for  our  research,  the  following  question  was  posed: 
What  Test  and  Evaluation  activity  is  required  for  Army  Programs  of  Records  (POR)? 

During  our  research,  we  examined  the  Defense  Acquisition  Framework,  DODI 
5000.1,  DODI  5000.2,  the  Defense  Acquisition  Guidebook  (DAG),  and  ATEC  Regulation 
73-1. 


T&E  is  defined  by  Test  and  Evaluation  Management  Guide  as  “the  process  by 
which  a  system  or  components  are  tested  and  results  analyzed  to  provide  performance 
related  information”  (Defense  Acquisition  University,  2015b).  The  information  gathered 
during  T&E  is  used  to  determine  the  operational  effectiveness,  suitability,  and 
survivability  of  systems  for  their  intended  use.  DAU  also  defines  the  terms  operational 
effectiveness,  suitability,  and  survivability. 

Operational  Effectiveness  (OE):  Measure  of  the  overall  ability  of  a 
system  to  accomplish  a  mission  when  used  by  representative  personnel 
in  the  environment  planned  or  expected  for  operational  employment  of 
the  system  considering  organization,  doctrine,  tactics,  supportability, 
survivability,  vulnerability,  and  threat.  (Defense  Acquisition  University, 

2015a) 

Survivability:  The  capability  of  a  system  or  its  crew  to  avoid  or 
withstand  a  manmade  hostile  environment  without  suffering  an 
abortive  impairment  of  its  ability  to  accomplish  its  designated  mission. 

(Defense  Acquisition  University,  2012) 

Operational  Suitability  (OS):  The  degree  to  which  a  system  can  be 
satisfactorily  placed  in  field  use  with  consideration  to  reliability, 
availability,  compatibility,  transportability,  interoperability,  wartime 
usage  rates,  maintainability,  safety,  human  factors,  habitability, 
manpower  supportability,  logistics  supportability,  documentation, 
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environmental  effects  and  training  requirements.  (Department  of 
Defense,  2012) 

One  of  the  first  things  a  new  program  must  establish  is  a  Test  and  Evaluation 
Strategy  (TES).  The  TES  (and  in  some  cases,  the  TEMP)  is  developed  during  the 
Materiel  Solution  Analysis  (MSA)  Phase  to  describe  how  T&E  will  be  used  throughout 
the  execution  of  program.  The  TES  is  written  in  accordance  with  the  Technology 
Development  Strategy  (TDS)  and  the  requirements  laid  out  in  the  Initial  Capabilities 
Document  (ICD).  The  TES  identifies  what  test  events  and  resources  are  required  during 
the  TDS  phase. 

The  TES  is  composed  of  four  major  sections.  Section  1  identifies  the  purpose  of 
the  program  (i.e.,  why  and  what  the  PM  is  building)  the  Materiel  Solutions  key 
capabilities  and  interfaces,  and  any  special  test  requirements  the  program  will  require. 
Section  2  of  the  TES  provides  the  roles  and  responsibilities  of  stakeholders  both  inside 
and  outside  the  program  office.  The  TES  lays  out  how  data  is  collected  and  disseminated 
during  DT  and  provides  an  integrated  test  schedule.  Section  3  of  the  TES  contains  the 
overall  evaluation  approaches  for  both  DT  and  OT.  As  a  part  of  this  process,  the 
Technology  Readiness  Levels  (TRLs)  and  prototype  testing  requirements  are  identified. 
This  section  also  identifies  how  and  when  Modeling  and  Simulation  (M&S)  will  occur. 
The  final  portion  of  this  section  identifies  the  test  limitations  and  required  future  test 
events.  Section  4  describes  the  resources  required  for  testing.  It  identifies  test  articles, 
instruments,  test  sites,  test  targets,  Operational  Force  Test  Support,  M&S  test  beds  and 
most  importantly,  test  funding  requirements  (Defense  Acquisition  University,  2015b). 

Once  the  TES  is  complete,  the  PM  submits  the  document  for  approval  to  the 
Component  Acquisition  Executive  (CAE).  If  the  system  is  on  the  OSD  T&E  oversight 
list,  the  PM  submits  the  TES  through  the  CAE  to  Office  of  the  Secretary  of  Defense 
(OSD)  T&E  for  approval.  As  the  program  moves  toward  Milestone  B  (MS-B),  the  TES 
evolves  into  the  TEMP  (Defense  Acquisition  University,  2015b). 

A  TEMP  is  required  for  all  Acquisition  Category  (ACAT)  I,  ACAT  IA,  II  and  III 
programs  and  any  under  OSD  T&E  oversight.  The  TEMP  must  do  five  things: 

•  Integrate  T&E  with  the  overall  Acquisition  Strategy 

18 


•  Reflect  the  user’s  requirements  and  describe  how  these  capability  needs 
will  be  tested  in  DT  and  OT 

•  Document  the  T&E  program  for  the  entire  life  cycle. 

•  Specify  personnel,  funding,  and  test  range  support  requirements 

•  Be  developed  prior  to  MS-B  and  updates  before  each  subsequent  Program 
Decision  Review  (Test  and  Evaluation  Master  Plan,  n.d.). 

Development  of  the  TEMP  is  the  responsibility  of  the  PM  and  the  Working-level 
Integrated  Product  Team  (WIPT),  both  of  which  will  sign  the  TEMP.  The  TEMP  should 
account  for  the  use  of  evolutionary  acquisition  that  will  not  only  reduce  risk  but  reduce 
the  burden  on  T&E.  The  cost  and  the  time  required  to  conduct  T&E  should  be  addressed 
in  the  acquisition  strategy  and  in  the  TEMP.  Accordingly,  the  TEMP  must  also  discuss: 

•  The  health  hazards  and  safety  issues  with  the  system 

•  Support  detailed  test  planning 

•  Provide  the  overall  structure  and  path  forward  for  all  future  testing,  M&S, 
schedules  and  resources  needed  to  meet  the  requirements  contained  within 
published  requirements  documents. 

Figure  3  provides  a  sample  TEMP  table  of  contents  from  the  Test  and  Evaluation 
Management  Guide  in  order  to  show  the  extensive  information  included  in  the  document 
(Defense  Acquisition  University,  2015b). 

There  are  three  types  of  T&E  activities  required  by  regulations  to  field  all  POR: 
DT&E,  OT&E,  and  Live  Fire  Test  and  Evaluation  (LFT&E).  The  TEMP  will  guide  each 
of  these  activities  and  clearly  lay  out  how  the  PM  intends  to  prove  that  the  system  is  safe, 
suitable,  and  effective. 

As  with  the  TES,  the  TEMP  is  submitted  for  approval  to  the  CAE  and  on  to  OSD 
T&E  as  required  by  the  program’s  level  of  oversight.  The  TEMP  is  tailored  to  each 
requirement  and  the  staffing  approval  process  is  different  based  on  the  ACAT  level.  The 
TEMP  is  a  representation  of  all  T&E  activities  that  bring  together  various  stakeholders. 
For  this  reason,  staffing  the  TEMP  can  be  the  most  difficult  and  time  consuming  of  all 
acquisition  documents. 
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Figure  3.  Example  TEMP  Major  Topics. 
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3.1  T&E  Strategy 
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Figure  3.1  -  Top-Level  Evaluation  Framework 
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4.1.4  Threat  Representation 
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From  Defense  Acquisition  University.  (2012).  Glossary  of  defense  acquisition  acronyms 
and  terms.  Retrieved  from  https://dap.dau.mil/glossary/Pages/Default.aspx 


Developmental  Testing  (DT)  is  conducted  throughout  the  acquisition  process  to 
determine  if  materiel  solutions  are  safe  and  will  meet  technical  performance  criteria.  In 
some  cases,  such  as  Chemical  and  Biological  Defense,  DT  will  also  begin  to  determine 
effectiveness,  survivability,  and  suitability.  DAU  defines  DT&E  as  “Any  testing  used  to 
assist  in  the  development  and  maturation  of  products,  product  elements,  or  manufacturing 
or  support  processes.  Any  engineering-type  test  used  to  verify  status  of  technical 
progress,  verify  that  design  risks  are  minimized,  substantiate  achievement  of  contract 
technical  performance,  and  certify  readiness  for  initial  OT”  (Defense  Acquisition 
University,  2015b). 
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DT  can  be  conducted  by  the  government  or  by  the  contractor.  Contractor  testing 
usually  occurs  at  the  vendor  facility  during  the  Engineering  and  Manufacturing 
Development  (EMD)  phase.  Testing  can  occur  on  prototypes,  board  models,  sub  systems, 
or  major  components.  A  brass  board  is  an  early  prototype  that  can  be  used  outside  the 
laboratory.  This  testing  is  designed  to  determine  the  maturity  of  the  production  process 
and  to  verify  that  components  are  operating  consistent  with  the  planned  performance. 
Once  complete,  this  data  is  approved  during  the  Technology  Readiness  Review.  This 
process  will  provide  the  government  team  the  information  needed  to  move  through  MS-C 
and  into  LRIP. 

Production  Qualification  Testing  (PQT)  occurs  after  MS-C  using  LRIP  assets. 
PQT  demonstrates  the  “integrity  of  the  system  design  over  the  operational  and 
environmental  range  in  the  specification”  (Defense  Acquisition  University,  2015b).  If 
software  is  involved,  testing  occurs  on  the  delivery  version.  PQT  informs  the  Milestone 
Decision  Authority  (MDA)  prior  to  making  the  FRP  decision.  DT&E  occurs  across  all 
phases  of  the  Acquisition  process  as  demonstrated  in  the  graphic  below  (Figure  4). 


Figure  4.  Examples  DT&E  Activities  across  Phases  of  the  Acquisition 

Life  Cycle. 
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From  R.  James,  personal  communication,  January  9,  2015. 
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Operational  Testing  and  Evaluation  is  required  by  Title  10,  United  States  Code, 
Section  2399  for  all  covered  systems.  A  covered  system  is  a  system  that  is  a  Major 
Defense  Acquisition  Program  (MDAP)  and  is  a  major  system  that  fulfills  a  mission  need. 
OT&E  is  further  defined  as  “the  field  test,  under  realistic  combat  conditions,  of  any  item 
of  (or  key  component  of)  weapons,  equipment,  or  munitions  for  the  purposes  of 
determining  the  effectiveness  and  suitability  of  the  weapons,  equipment,  or  munitions  for 
use  in  combat  by  typical  military  users;  and  the  evaluation  of  the  results  of  such  test”  (10 
U.S.  Code  Section  2399). 

Operational  Testing  (OT),  as  stated  by  the  Test  and  Evaluation  Management 
Guide  (Defense  Acquisition  University,  2015b),  is  “the  degree  of  satisfaction  of  the 
user’s  requirements  expressed  as  operational  effectiveness  and  operational  suitability  of 
the  new  system.”  In  the  case  of  Chemical,  Biological,  Radiological,  and  Nuclear  (CBRN) 
equipment,  this  may  also  be  determined  in  part,  through  DT.  The  level  of  operational 
effectiveness  determines  the  answer  to  the  basic  question:  will  a  system  work  in  its 
intended  environment?  Effectiveness  is  the  measure  of  performance  of  a  system  when  it 
is  used  by  end  users  in  an  operational  environment  using  current  Techniques,  Tactics  and 
Procedures  (TTP),  doctrine,  and  threat  scenarios. 

Suitability  is  used  to  measure  the  interoperability  of  the  system  when  considered 
from  a  transportability  and  Reliability,  Availability,  and  Maintainability  (RAM)  stand 
point.  OT  may  not  extend  long  enough  to  fully  determine  RAM.  Results  from  OT  and  DT 
can  be  combined  to  gather  sufficient  RAM  data.  Suitability  ensures  that  the  system  can 
be  deployed,  used  and  maintained  in  the  field. 

Survivability  is  the  measurement  of  a  system’s  vulnerability  and  survivability. 
These  are  measured  through  LFT&E.  LFT&E  is  required  by  all  ‘covered’  acquisition 
programs.  For  LFT&E,  the  term  ‘covered’  applies  to  a  system  that  users  occupy  in 
combat,  any  conventional  munitions  and  missile  program,  a  conventional  munitions 
program  where  the  acquisition  of  a  million  rounds  is  planned,  any  modification  to  a 
current  covered  system,  or  any  system  designated  for  oversight  by  Director  Operational 
Test  &  Evaluation  (DOT&E).  LFT&E  is  conducted  to  measure  how  well  the  system 

responds  to  live  fire  and  also  how  well  the  crew  survives. 
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There  are  some  key  facts  to  remember  regarding  OT&E. 

•  OT&E  is  conducted  throughout  all  phases  of  the  acquisition  process. 

•  The  DOT&E  is  the  OT&E  approval  authority  for  all  MDAPs 

•  OT&E  is  conducted  by  an  independent  Operational  Test  Authority  (OTA). 

In  summary,  T&E  informs  the  MDA  of  the  system’s  ability  to  meet  the 
requirements  as  prescribed  by  JCIDS  requirements  documents.  All  acquisition  programs, 
regardless  of  AC  AT,  require  independent  T&E.  The  level  of  testing  will  be  determined 
by  the  MDA  and  the  OTA.  The  TES  will  determine  the  personnel  and  logistics  of  T&E. 
Once  the  program  transitions  from  MS  A  to  MS  B,  the  TES  will  evolve  into  the  TEMP. 
The  TEMP  will  then  layout  the  T&E  process  the  program  will  follow.  DT  testing  will 
determine  if  the  system  performs  as  required,  meets  user  requirements,  and  if  a  system  is 
ready  for  OT.  OT,  which  is  performed  by  the  government  using  an  independent  OTA, 
will  determine  if  a  system  is  effective,  suitable,  and  survivable.  Additionally,  ‘covered’ 
systems  require  LFT&E  to  verify  the  survivability  and  lethality. 

2.  T&E  Requirements  for  Rapid  Acquisition 

When  the  Army  has  a  capability  gap  that  cannot  wait  for  fulfillment  by  the  normal 
acquisition  process  or  if  the  gap  is  non-enduring,  the  Army  uses  RA  to  close  the  gap.  In 
order  for  a  commander  to  utilize  the  solution,  it  must  be  evaluated  by  ATEC.ATEC  will 
determine  its  capabilities  and  limitations  as  well  as  ensuring  it  is  safe  for  Soldiers  to  use. 

The  Army  ONS  process  is  used  when  an  operational  commander  identifies  a 
capability  gap  and  is  either  deployed,  deploying,  or  a  high  priority  unit.  According  to  AR 
71-9,  there  are  three  types  of  ONS:  those  for  a  deployed  unit,  those  for  deploying  unit, 
and  an  ONS  from  other  units.  This  is  also  how  priorities  are  established  for  each  ONS. 
The  highest  priority  ONS  is  a  deployed  unit  in  a  named  operation.  The  second  highest 
priority  ONS  is  a  unit  deploying  to  a  named  operation.  The  lowest  priority  ONS  is  from 
all  other  units  that  have  recognized  a  capability  gap  that  requires  a  materiel  solution 
(Department  of  the  Army,  2009d). 


23 


Regardless  of  the  type,  Commanders  follow  the  same  ONS  process.  Once 
initiated,  an  ONS  formally  enters  the  process  through  the  ECOP.  The  ECOP  database  is 
located  at  https://www.ecop.armv.smil.mil  on  the  Secret  Internet  Protocol  Router 
Network  (SIPRNet).  ECOP  requires  access  permission  that  can  usually  be  granted  in  a 
few  days.  The  user  must  first  identify  the  gap  and  provide  information  on  what  missions 
cannot  be  accomplished.  Next,  the  user  provides  justification  for  the  requirement  by 
providing  rationale  on  what  impact  of  not  fulfilling  the  ONS  will  have  on  mission 
accomplishment.  The  unit  must  also  demonstrate  what  efforts  were  used  to  meet  the 
mission  requirement  with  existing  equipment  and  organization.  The  unit  will  then  define 
how  the  equipment  will  be  used,  who  will  use  it,  and  the  Organizational  Concept  for 
employment.  The  unit  will  define  how  many  systems  are  required,  how  they  will 
complete  training  on  the  system,  additional  support  needed,  and  recommend  a  source  of 
supply  or  solution  set.  Once  the  request  is  complete,  it  can  flow  through  the  ONS/ 
Equipment  Sourcing  Document  (ESD)  approval  process.  Figure  5  from  the  ECOP  Pocket 
guide  illustrates  the  ONS/ESD  staffing  process  (Department  of  the  Army,  n.d.).  If  the 
unit  is  deployed,  the  request  is  routed  through  the  deployed  unit’s  chain  of  command  to 
the  ASCC.  Non-deployed  units  will  submit  requests  through  their  major  command.  If  the 
ONS  is  approved  by  the  respective  ASCC,  it  moves  on  to  HQDA  G3/5/7  for  validation.  If 
validated,  the  ONS  moves  to  HQDA  G8  for  resourcing.  If  resourcing  is  available,  the 
validated  request  is  forwarded  with  funding  to  the  acquisition  community  for  action.  If 
funding  is  not  available,  the  requirement,  must  appear  before  the  Army  Requirements  and 
Resourcing  Board  (AR2B).  The  AR2B  is  responsible  for  prioritizing  and  validating 
resources  for  current  requirements  and  the  reprioritization  of  resources  as  required.  The 
AR2B  primarily  deals  with  resourcing  within  the  year  of  execution  and  is  also 
responsible  for  notifying  Congress  on  resource  changes  that  are  required  by  law.  Once 
funding  is  approved  by  the  Army  Budget  Office  (ABO),  it  is  assigned  to  a  PM.  The  PM 
accepting  the  requirement  and  resourcing,  moves  forward  with  an  acquisition  plan  to  fill 
the  capability  gap  (Department  of  the  Army,  2009). 
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Figure  5.  ECOP  ONS/ESD  Staffing  Process. 
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During  the  DA  3/5/7  evaluation  period,  the  Army  Staff  evaluates  the  ONS  for 
broader  applications  across  the  department.  If  the  determination  is  made  that  the  ONS 
affects  a  broader  audience,  DA  3/5/7  will  release  a  Directed  Requirement. 

Directed  Requirements  (DR)  are  authored  by  HQDA  G-3/5/7  Capabilities 
Integration  Division  (DAMO-CI)  and  approved  by  the  VCSA.  Once  approved,  the  DR 
appears  in  front  of  the  AR2B  for  resourcing.  DRs  cannot  be  used  to  develop  new 
technology.  The  DR  will  only  provide  capability  outside  the  traditional  JCIDS  process 
and  only  if  the  unfilled  capability  gap  endangers  lives  or  seriously  impacts  mission 
accomplishment. 

Once  the  PM  receives  an  ONS  or  DR  and  resources,  the  process  of  fulfilling  the 
urgent  requirement  begins.  To  provide  the  required  capability,  there  are  several  activities 
that  must  occur.  The  PM,  working  with  ATEC,  the  user,  and  the  MDA,  must  produce 
documentation  as  required  by  AR  700-142.  These  documents  consist  of  a  Safety 
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Confirmation,  a  Capabilities  and  Limitation  Report,  a  User  Acceptance  Memorandum, 
and  finally,  an  UMR  is  approved  by  the  MDA  (Department  of  the  Army,  2013c). 

The  PM  is  responsible  for  funding  and  requesting  support  from  ATEC.  Once 
ATEC  receives  the  request,  it  is  assigned  to  an  ATEC  System  Team  (AST).  Once 
assigned  to  an  AST,  a  chair  is  assigned  with  the  primary  responsibility  of  leading  the 
T&E  effort.  The  AST  Chair  will  determine  how  the  ONS/DR  will  be  evaluated.  The  AST 
will  use  the  ATEC  Rapid  Acquisition  Initiative  flow  chart  (shown  in  Figure  6)  to  assist  in 
the  management  of  the  test  effort  (Army  Test  and  Evaluation  Command,  2004). 

Once  the  PM  has  provided  a  solution  and  the  AST  has  developed  a  test  plan,  the 
effort  can  be  moved  into  the  evaluation  portion  of  the  process.  A  Safety  Release  is 
required  prior  to  a  test  event  and  once  testing  is  complete,  a  Safety  Confirmation  is 
required  to  field  equipment. 

A  Safety  Release  is  required  prior  to  placing  equipment  into  the  hands  of  soldiers. 
AR  385-10  states  “A  safety  release  must  be  obtained  from  Headquarters,  ATEC  or  in 
accordance  with  guidance  provided  in  DA  PAM  385-16  whenever  solders  and  involved 
with  an  event”  (Department  of  the  Army,  2013a). 

The  Safety  Release  is  developed  and  written  by  the  U.S.  Army  Evaluation  Center 
(AEC)  Test  Manager  having  primary  responsibility.  The  Safety  Release  has  several  key 
parts.  First,  the  release  covers  a  specific  event  i.e.,  Operational  Assessment  (OA), 
Limited  User  Test  (LUT),  or  OT.  It  also  covers  a  specific  time  period  with  definitive  start 
date  and  has  an  expiration  date.  The  Safety  Release  should  identify  the  test  participants 
that  will  use  the  equipment,  the  level  of  risk  the  soldiers  will  be  exposed,  and  the  overall 
safety  risk  of  the  system.  The  safety  release  addresses  each  piece  of  equipment  and  each 
component,  evaluating  against  the  requirements  laid  out  in  Military  Standard  (MIL-STD) 
882E  for  risk.  It  is  important  to  remember  a  safety  release  is  a  temporary  document  used 
for  a  specific  event. 
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Figure  6.  RAI  and  Urgent  Materiel  Release  Process. 
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From  U.S.  Army  Test  and  Evaluation  Command.  (2013).  System  test  and  evaluation 
policy.  ATEC  Regulation  73-1.  Washington,  DC:  U.S.  Army  Test  and  Evaluation 

Command. 
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A  Safety  Confirmation  is  used  during  the  normal  course  of  acquisition  and  is 
required  by  AR  385-10  prior  to  each  Milestone.  It  is  also  required  during  RA  prior  to  an 
UMR  decision.  AR  385-10  defines  a  Safety  Confirmation  as  “A  separate  document  or 
part  of  the  OTA  evaluation  report,  OTA  Milestone  evaluation  report,  OTA  Assessment 
report,  or  Capabilities  and  Limitations  Report  (CLR)  that  provides  the  DT  or  OT  agency 
findings  and  conclusions,  and  states  whether  the  specified  safety  requirements  are  met” 
(Department  of  the  Army,  2013b).  The  purpose  of  this  document  is  to  evaluate  the  risk 
that  the  system  poses  to  soldiers,  ensure  those  risks  are  mitigated  or  removed,  and  in 
cases  where  residual  risks  remain,  provide  the  user  with  a  risk  assessment  to  allow  the 
user  to  manage  the  risk.  ATEC  collects  this  information  through  the  evaluation  of 
Technical  Manuals,  DT  results,  and  observations  during  OAs.  The  Safety  Confirmation  is 
one  of  the  key  documents  that  is  required  in  order  to  receive  an  UMR  from  a  MDA. 
Table  1  shows  when  a  Safety  Release  or  Confirmation  is  required. 


Table  1.  Safety  Release,  Safety  Confirmation  Comparison. 
Safety  Release/Confirmation  Matrix 

(In  Accordance  with  AR  70-1  &  73-1) 


From  U.S.  Department  of  the  Army.  (2015).  Type  classification,  materiel  release, 
fielding  and  transfer.  Army  Regulation  700-142.  Washington,  DC:  U.S.  Department  of 

the  Army. 


A  CLR  is  the  culmination  of  DT  and  OT  events  that  occurred  informing  the 
gaining  command  and  the  materiel  release  authority  what  the  system  can  and  cannot  do. 
The  AST  Chair  and  the  PM  evaluate  the  requirement  as  written  in  the  ONS  or  DR.  After 
evaluation,  the  AST  Chair  develops  a  Test  Concept  Plan,  which  is  approved  by  ATEC 
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CG  and  presented  to  the  PM  for  funding.  DT  to  support  the  CLR  is  conducted  by  various 
Test  centers  and  OT  is  conducted  by  Operational  Test  Command  (OTC).  The  AST  chair 
and  the  PM  or  program  sponsor  will  determine  the  type  of  operational  testing  based  on 
the  complexity  of  the  system.  At  a  minimum,  some  type  of  OA  will  occur  to  place  the 
capability  in  the  hands  of  soldiers.  The  evaluation  will  provide  key  inputs  into  the  CLR. 

The  CLR  consists  of  nine  key  areas:  purpose,  executive  summary,  mission  need, 
system  description,  data  sources,  test  limitations,  observations,  employment 
considerations,  and  recommendations. 

The  Purpose  paragraph  helps  explain  the  reason  for  the  evaluation  providing 
context  to  both  the  user  and  acquisition  community.  The  executive  summary  provides  a 
brief  overview  of  the  CLR  and  its  contents.  The  mission  need  paragraph  lays  out  what 
capability  is  required  and  should  reference  the  ONS  number  and  identified  unit  or  the 
pertinent  DR.  The  system  description  paragraph  lays  out  the  system  and  how  it  is 
intended  to  work.  It  should  also  state  how  it  should  be  used  in  its  intended  environment. 
The  data  sources  and  test  limitations  paragraphs  tell  what  tests  were  conducted,  both  OT 
and  DT,  and  what  limitations  where  present  during  testing.  The  observation  paragraph 
lists  all  the  capabilities  of  the  system  and  the  current  warfighter  gaps  the  system  fills.  It 
includes  all  capabilities  and  limitations  observations  as  well  as  safety,  interoperability, 
training,  supportability,  and  survivability  observations.  The  CLR  also  includes 
recommendations  to  both  the  user  on  how  to  employ  the  capability,  and  to  the  acquisition 
community  on  how  to  make  the  system  more  suitable  and  effective. 

Once  the  CLR  and  the  Safety  Confirmation  are  in  place,  the  PM  must  gain 
acceptance  of  the  capability  from  the  Gaining  Command.  The  Gaining  Command  will 
provide  an  acceptance  memorandum,  usually  signed  by  a  general  officer  or  a  senior 
executive  service  civilian  equivalent.  The  memo  should  cover  the  items  accepted  and  the 
quantity  accepted.  If  the  acceptance  memo  is  for  a  Conditional  Materiel  Release,  the  user 
acceptance  memo  will  cover  the  duration  of  the  acceptance.  Once  accepted,  the  PM  is 
ready  to  seek  an  UMR  from  the  MDA. 
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A  Materiel  Release  (MR)  is  granted  much  in  the  same  way  a  milestone  decision  is 
granted.  The  MR  authority  will  review  the  documentation  for  completeness,  then  make  a 
determination  to  grant  the  MR.  The  procedure  for  all  MRs  (Full,  Training,  Conditional, 
or  Urgent)  are  governed  by  AR  700-142.  The  authority  to  grant  a  materiel  release 
depends  on  the  materiel  involved  (Department  of  the  Army,  2013c). 

MR  authority  is  granted  in  accordance  with  AR  700-142.  There  are  several 
release  authorities  that  can  grant  a  MR  depending  on  the  type  of  materiel  involved. 

The  CLR  and  the  Safety  Confirmation  are  released  simultaneously  and  once 
coupled  with  a  gaining  command  acceptance  memo,  are  used  by  the  MDA  to  grant  a 
UMR.  The  UMR  is  granted  for  a  specific  event,  specific  quantities,  and  to  specific 
commanders.  Once  the  UMR  is  granted,  the  PM  can  then  equip  the  user  with  the 
prescribed  capability.  After  fielding,  the  capability  is  monitored,  with  ATEC  updating  the 
CLR  as  needed.  HQDA  and  TRADOC  will  determine  if  the  fielded  capability  will 
become  enduring  and  transition  to  a  program  of  record  or  disposed  of  once  the  capability 
is  not  required. 


3.  UMR  and  FMR  Comparison 

The  following  tables  (Tables  2-5)  compare  the  different  activities  required  for  a 
POR  and  associated  FMR  versus  a  shorter  term  RA  and  associated  UMR.  Some  of  the 
FMR  requirements  may  also  factor  into  PEO  leadership  taking  on  the  responsibility  to 
field  an  RA,  as  well  as  a  GC  being  satisfied  with  and  accepting  the  new  capability.  These 
are  notated  with  a  “PD”  for  “program  dependent”  since  not  all  programs  will  require  the 
same  T&E  efforts.  However,  if  certain  activities  usually  required  for  FMR  are  required 
for  GC  acceptance  under  an  RA  effort,  they  may  be  an  abbreviated  version  or  may  also 
still  show  that  risk  exists  in  certain  areas,  but  may  be  acceptable  to  the  GC.  Additionally, 
an  area  of  great  variation  may  exist  within  the  suitability  category  where  an  FMR 
requires  adherence  to  the  detailed  and  long-term  program  TEMP  and  TES,  and  an  UMR 
addressed  the  suitability  question  with  a  CLR  that  the  AST  chair  defines  in  a  program 
specific  manner. 
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Table  2.  Materiel  Release  Safe  Hazards  Release  Requirements. 


FMR  -  Safe  Hazards 

FMR 

UMR 

Supporting  safety  office  certification 

X 

PD 

TSG  HHA 

X 

Environmental  statement 

X 

Airworthiness  statement 

X 

PD 

SSRA  for  residual  hazards 

X 

Surface  or  Weapon  danger  zone 

X 

Final  hazard  classification 

X 

NRC  license 

X 

Army  Fuze  Safety  Review  Board  Certification 

X 

Energetic  materiels  qualification 

X 

Ignition  System  Safety  Review  Board  Certification 
and  Standardization  Agreement 

X 

Safety  review  of  technical  manuals 

X 

Results  of  safety  inspections  and  analyses 

X 

PD 

Software  safety  statement 

X 

After  U.S.  Department  of  the  Army.  (2015).  Type  classification,  materiel  release,  fielding 
and  transfer.  Army  Regulation  700-142.  Washington,  DC:  U.S.  Department  of  the 

Army.. 


Table  3.  Materiel  Release  Supportability  Requirements. 


FMR  -  Supportability 

FMR 

UMR 

Supportability  certification — will  address  support 

X 

PD 

materiel,  end  item,  and  software 

USATA  supportability  statement  on  TMDE  or  ATE 

X 

TC  designation 

X 

SDDC  TEA  transportability 

X 

Army  logistician  assessment 

X 

PD 

Supporting  statements  for  COEI  and  ASOIE 

X 

PD 

Software  supportability  statement 

X 

PD 

After  U.S.  Department  of  the  Army.  (2015).  Type  classification,  materiel  release,  fielding 
and  transfer.  Army  Regulation  700-142.  Washington,  DC:  U.S.  Department  of  the 

Army. 
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Table  4.  Materiel  Release  Suitability  Requirements. 


FMR  -  Suitability 

FMR 

UMR 

ATEC  MR  position  memorandum 

X 

PD 

ATEC  OMAR  or  OER 

X 

PD 

CIO/G-6  AIC  statement  based  upon  AIC  completion 

X 

PD 

Certificate  of  Networthiness 

X 

PD 

DIACAP  certification  statement 

X 

Communications  Security  Logistics  Activity 
statement  for  COMSEC  supportability 

X 

CAPDEV  training  assessment  (statement  of  adequacy 
of  institutional  training  support) 

X 

Software  suitability  statement 

X 

Quality,  reliability,  availability,  and  maintainability 
statement,  including  service  or  shelf  life  assurance, 
Ammunition  Stockpile  Reliability  Program,  and 
ammunition  surveillance  procedures 

X 

PD 

After  U.S.  Department  of  the  Army.  (2015).  Type  classification,  materiel  release,  fielding 
and  transfer.  Army  Regulation  700-142.  Washington,  DC:  U.S.  Department  of  the 

Army. 


Table  5.  Materiel  Release  RA  UMR  Requirements 


UMR 

FMR 

UMR 

User  requested/HQDA  directed 

X 

Safety  and  Health  data  sheet  with  a  risk  assessment 
for  the  materiel  system 

X 

Air  worthiness  statement  (if  applicable) 

X 

EOD  supportability  statement  (if  applicable) 

X 

PM  Request  for  acceptance  from  the  GC  or  requestor 

X 

GC  acceptance  statement 

X 

After  U.S.  Department  of  the  Army.  (2015).  Type  classification,  materiel  release,  fielding 
and  transfer.  Army  Regulation  700-142.  Washington,  DC:  U.S.  Department  of  the 

Army.  . 


B.  PROGRAM  RESEARCH  DATA 

The  data  presented  in  this  section  provides  the  reader  with  two  specific  examples 
of  programs  that  have  started  as  RA,  made  the  transition  to  POR,  and  the  T&E  activities 
required  to  make  that  transition. 
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1. 


Dismounted  Reconnaissance  Sets  Kits  and  Outfits 


The  DR  SKO  is  an  ACT  III  program  assigned  to  the  Joint  Project  Manager  for 
Nuclear,  Biological  and  Chemical  Contamination  Avoidance  (JPM  NBC  CA)  and  the 
JPEO  CBD.  The  JPEO  CBD  is  responsible  for  the  “Research,  Development,  Acquisition 
Fielding  and  Life-Cycle  Support  of  Chemical,  Biological,  Radiological  and  Nuclear 
Defense  Equipment,  Medical  Countermeasures  and  Installation  and  Force  Protection 
Integrated  Capabilities  Supporting  the  National  Strategies”  (Joint  Project  Manager  for 
Nuclear,  Biological  and  Chemical  Contamination  Avoidance  [JPEO  CBD],  n.d.,  Mission 
Statement).  JPM  NBC  CA  is  responsible  for  the  development,  production,  integration, 
testing  and  fielding  of  NBC  detection,  obscuration,  and  reconnaissance  systems  (JPM 
NBC  CA  (n.d.).  Mission  Statement).  The  acquisition  objective  for  this  program  was  for 
440  systems  at  a  cost  upward  of  $500  million. 

The  DR  SKO  is  used  by  all  services  to  conduct  dismounted  CBRN 
reconnaissance.  The  system  is  a  mixture  of  Commercial  off  the  Shelf  (COTS)  and 
Government  off  the  Shelf  (GOTS)  detection,  sampling,  personnel  protection,  and 
decontamination  hardware  (overview  shown  in  Figure  7).  The  system  is  designed  for 
world- wide  deployment,  packaged  to  meet  individual  service  needs. 

The  primary  mission  of  the  DR  SKO  is  to  identify  the  presence  of  CBRN  and 
Weapons  of  Mass  Destruction  (WMD)  materiels  (and  their  precursors),  and  can 
characterize  the  environment.  Information  gathered  by  this  system  can  provide  critical 
information  to  the  commander  on  the  presence  and  use  of  CBRN  weapons.  In  his 
government  blog,  the  DR  SKO  Systems  Manager,  Mr.  Edward  Conley,  described  the 
system  as  one  that,  “provides  a  modular  baseline  suite  of  modem  detection, 
identification,  and  protection  equipment  to  support  the  ever  changing  threat  environment, 
especially  for  toxic  industrial  hazards”  (Edgewood  ChemBioCenter,  2012).  The  genesis 
of  the  DR  SKO  came  from  an  ONS  generated  by  units  experiencing  capability  gaps 
during  OEF.  U.S.  forces  encountered  multiple  threats  from  Toxic  Industrial  Chemicals 
(TIC)  and  Toxic  Industrial  Materiels  (TIM). 


33 


At  the  time,  traditional  CBRN  organizations  were  equipped  to  conduct  CBRN 
Reconnaissance  against  traditional  warfare  agents  but  lacked  the  ability  to  conduct  site 
assessment,  conduct  confined  space  operations,  identify  TIC/TIMs,  and  had  limited 
capability  inside  buildings  in  urban  environments.  After  fulfillment  of  the  ONS,  the  Joint 
Force,  understanding  the  value  of  this  new  capability,  submitted  a  JUONS  to  increase  the 
capability  from  16  to  40  systems.  This  evolution  in  capability  over  time  is  illustrated  in 
Figure  8.  With  demand  for  this  capability  increasing,  Army  Combat  Developers  began 
development  of  JCIDS  documents. 

Figure  7.  DR  SKO  Layout. 


From  JPEO  CBD.  (2014).  Dismounted  reconnaissance  sets  kits  and  outfits  [Fact  sheet]. 

Retrieved  from 

http://jacks.jpeocbd.army.mil/Public/FactSheetProvider.ashx?productId=447 


The  early  acquisition  strategy  attempted  to  capitalize  on  the  heavy  use  of  existing 
COTS  or  GOTS  technology  to  enter  into  a  MS-C  decision  with  a  production  CPD.  Ms. 
Anne  Hise,  the  Lead  System  Engineer  (Edgewood  ChemBioCenter,  2012)  stated, 
“Originally,  the  strategy  included  initial  entry  into  the  acquisition  process  at  the  Low- 
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Rate  Initial  Production  decision.  The  lack  of  independently  verified  data  to  support 
evaluation  of  key  performance  parameters  drove  a  change  in  strategy”  The  acquisition 
team,  unable  to  verify  many  manufacturer  claims  and  match  those  claims  to  Key 
Performance  Parameters  (KPP)  and  Key  System  Attributes  (KSA)  laid  out  in  the  CPD, 
re-grouped,  moved  the  program  back  to  a  MS-B,  and  converted  the  CPD  back  to  a  CDD. 


Figure  8.  DR  SKO  Pre  Program  Evolution. 
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TIC  POE  -  Toxic  Industrial  Chemical  Protection  Detection  Equipment  ONS/JUONS  -  Operational  Needs  Statement  I  Joint  Urgent  Operational  Needs  Statement 
CCMRF  -  CBRN  Consequence  Management  Response  Force  CDD  -  Capabl  ty  Development  Document 

DR-SKO  -  Dsmounted  Reconnaissance  Sets,  Kits  or  Outfits  CPD  -  Capability  Production  Document 

MS -SKO  -  Monitoring  and  Survey  Sets.  Kits  or  Outfits 


From  JPEO  CBD.  (2009).  DOD  chemical  biological  defense  advanced  planning  briefing 
for  industry  (APB I).  Retrieved  http://www.jpeocbd.osd.mil/packs/Default.aspx?pg=851 


The  Joint  Requirements  Office  for  CBRN  Defense  authored  a  CDD  and  the  JPEO 
CBD,  the  MDA  for  this  program,  issued  a  MDD  on  July  6,  2010.  The  MDA  approved  the 
DR  SKO  for  entrance  into  the  acquisition  cycle  at  MS-B  and  also  approved  the 
Acquisition  Decision  Memorandum  (ADM)  on  March  30,  2011.  Once  approved,  the  DR 
SKO  could  enter  into  the  EMD  phase.  Two  major  test  events  occurred  during  EMD: 
Engineering  Developmental  Testing  (EDT)  and  OAs. 
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EDT  occurred  at  multiple  Test  Centers  across  the  United  States.  Chemical  testing 
took  place  at  Dug  way  Proving  Grounds  and  the  Edgewood  Chemical  Biological  Center, 
trace  explosive  testing  took  place  at  Indian  Head,  MD  and  the  Electronic  Proving 
Ground,  and  radiation  testing  occurred  at  White  Sands  Missile  Range.  The  EDT  validated 
equipment  to  verify  manufacturer  claims  and  to  meet  the  requirements  of  the  CDD. 

Two  OAs  where  conducted:  one  for  ground  based  operations  and  one  for 
maritime  operations.  The  United  States  Marine  Corps  (USMC)  tested  the  ground  based 
version  in  an  urban  environment  at  Ft.  Hood,  Texas,  and  the  United  States  Navy  (USN) 
tested  the  maritime  version  under  an  excursion  off  the  coast  of  North  Carolina. 

The  EDT  and  OAs  provided  the  required  data  to  move  DR  SKO  to  the  next 
acquisition  phase.  A  CPD  was  developed  and  a  MS-C  decision  took  place  during  the 
second  quarter  of  2013.  With  the  MS-C  decision  in  hand,  the  DR  SKO  program  moved 
toward  the  FRP  Decision. 

To  meet  the  requirements  of  the  FRP  decision,  additional  testing  was  conducted. 
The  DR  SKO  program  conducted  Performance  Qualification  Testing,  Logistics  and 
Maintainability  Demonstration  (LMD),  and  Multiservice  Operational  Testing  (MOT). 

DR  SKO  tested  three  areas  as  part  of  PQT:  Transportability  and  Safety,  System 
level  RAM  and  Vapor  Sampling  Collection  Demonstration.  Transportability  testing  was 
used  to  assess  system  safety  and  verify  modification  made  post  EDT  (Department  of 
Defense,  2008).  The  systems  were  exposed  to  drop  testing,  on  course  vibration,  rail 
impact,  and  a  host  of  environmental  testing.  RAM  testing  operated  components  of  the 
system  for  a  total  of  515  hours.  Certified  civilian  operators  conducted  operations  for  14.5 
hour  mission  cycles  consistent  with  the  DR  SKO  Operational  Mode  Summary/Mission 
Profile  (OMS/MP)  (JPM  CA,  December  2012).  The  final  PQT  event  was  the  Vapor 
Sampling  Collection  Demonstration.  This  event  measured  the  ability  of  the  equipment  to 
collect  a  sample  and  maintain  the  sample  until  delivered  to  a  laboratory  for  analysis. 

The  DR  SKO  conducted  a  Logistics  and  Maintainability  Demonstration  assessing 
and  validating  the  maintainability,  repair  ability  and  the  system  logistics.  The  demo  lasted 
one  month  in  order  to  test  systems  configured  for  the  Army,  USN,  and  USMC.  The 
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purpose  of  the  Logistics  Demonstration  was  to  determine  system  supportability,  validate 
the  Technical  Manual,  verify  the  Maintenance  Allocation  Chart,  and  ensure  the 
completeness  of  the  System  Support  Package.  The  Maintainability  Demonstration 
measured  the  adequacy  of  built-in  test,  the  tool  kit,  the  provided  test  kit,  and  the 
measurement  of  both  corrective  and  preventive  maintenance  times. 

The  final  test  event  was  the  MOT.  The  MOT  provided  data  to  determine  the 
effectiveness,  suitability,  and  survivability  of  the  DR  SKO.  The  MOT  was  evaluated  by 
the  U.S.  Army  Operational  Test  Command  (OTC)  against  realistic  operational  scenarios. 
Each  service  conducted  OT  in  environments  and  scenarios  relevant  to  their  mission. 

Once  testing  was  complete,  the  MDA  granted  an  FRP  decision  during  the  second 
quarter  of  Fiscal  Year  2014  and  full  rate  production  started  immediately.  The  program  is 
still  in  production  with  Initial  Operational  Capability  (IOC)  for  all  services  scheduled  for 
third  quarter  Fiscal  Year  2016.  Figure  9  shows  the  DR  SKO  Integrated  Master  Schedule 
(IMS)  from  the  DR  SKO  TEMP. 

Figure  9.  DR  SKO  IMS  from  DR  SKO  TEMP. 


From  JPM  CA.  (2012).  Dismounted  reconnaissance  sets,  kits,  and  outfits  (DR  SKO)  test 
and  evaluation  master  plan  (TEMP).  Aberdeen  Proving  Grounds,  MD:  JPEOCBD. 
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2.  Counter  Remote  Control  Improvised  Explosive  Device  Electronic 
Warfare 

The  primary  Army  vehicle  mounted  Counter  Remote  Control  Improvised 
Explosive  Device  Electronic  Warfare  system  is  named  Duke  (AN/VLQ-12).  The 
following  information  is  derived  from  the  CREW  Duke  SEP  (PEO  IEW&S,  2013)  and 
TEMP  (PEO  IEW&S,  2012). 

The  CREW  Duke  (or  Duke  Countermeasures  Set)  is  a  second  generation 
Electronic  Countermeasure  (ECM)  set  used  to  jam  Radio  Frequency  (RF)  threats  and  is 
comprised  of  a  complete,  end-to-end  RF  system  (PEO  IEW&S,  2012,  p.  1-2).  The 
system  chassis  is  self-contained  with  external  power  connectors  for  use  with  batteries, 
vehicle  power,  or  generators,  and  performs  the  receiving,  processing,  and  response 
transmitting  function  (PEO  IEW&S,  2012,  pp.  1-2).  All  of  the  end  user  controls 
necessary  to  operate  the  system  are  user-accessible  with  simple  switches  to  change 
operational  modes  (PEO  IEW&S,  2013,  p.  14). 

Duke  systems  are  needed  to  provide  commanders  with  the  capability  to 
meet  urgent  mission  requirements,  provide  vital  force  protection 
against  RCIED  threats,  and  reduce  the  risk  of  serious  injury  and  loss  of 
life  to  the  Warfighter.  During  full  spectrum  operations,  the  Duke 
system  is  a  combat  power  enabler.  The  CREW  Duke  system  provides 
commanders  the  freedom  of  movement/action  required  to  concentrate 
and  disperse  combat  power  to  keep  the  enemy  at  a  disadvantage.  (PEO 
IEW&S,  2012,  p.  1-1) 

The  system  protects  ground  forces  operating  in  convoys  or  single  vehicle 
operations  against  signals  that  trigger  RCIEDs. 

The  CREW  Duke  system  is  represented  by  two  configurations  (V2  and  V3)  and 
are  deployed  in  theater  on  all  designated  tactical  and  non-tactical  vehicles.  The  system 
has  demonstrated  adequate  and  reliable  protection  against  priority  RCIED  threats  (PEO 
IEW&S,  2012,  pp.  1-1-  1-2). 

The  CREW  Duke  TEMP  details  the  need  for  CREW  growing  out  of  the  surge  of 
U.S.  casualties  as  a  result  of  roadside  bombs,  or  Improvised  Explosive  Devices  (IEDs), 
during  OIF  and  OEF  (2012).  “IEDs  have  been  the  leading  cause  of  U.S.  Casualties,  and 
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RCIEDs  represent  a  very  large  portion  of  these  casualties”  (PEO  IEW&S,  2012,  p.  1-5). 
CREW  systems  in  general  (and  specifically  CREW  Duke  for  the  Army)  have 
dramatically  reduced  the  effectiveness  of  RCIEDs  (PEO  IEW&S,  2012,  p.  1-5).  As 
CREW  Duke  systems  were  rapidly  deployed,  there  was  a  significant  reduction  in  RCIED 
related  casualties. 

Constant  evolution  of  the  RCIED  threat  required  specific  changes  to  the  Duke 
system  in  order  to  maintain  relevancy.  Multiple  JUONS  identified  a  critical  need  for 
increased  capabilities  that  were  provided  by  upgrading  the  Duke  VI  to  the  Duke  V2  and, 
finally,  to  the  Duke  V3  configuration.  In  August  2010,  the  CREW  Duke  system  was 
designated  as  an  ACAT  II  Program  (PEO  IEW&S,  2012,  pp.  1-6). 

As  an  urgent  need  RA,  the  CREW  Duke  system  was  specified  by  a 
Performance  Based  Specification  (PBS)  via  a  competitive  bid  process. 

In  order  to  maximize  flexibility  for  the  respondents,  the  System 
Architecture  and  Interface  Control  were  not  specified  by  the 
Government.  Instead,  the  respondents  were  free  to  use  the  architecture 
“as  is”  and  interface  control  method  which  would  best  allow  them  to 
meet  the  requirements  of  the  PBS.  An  interface  control  document  was 
required  to  be  submitted  by  the  Original  Equipment  Manufacturer 
(OEM)  to  define  the  physical  interfaces.  This  defined  interface  control 
is  a  governing  feature  for  any  future  upgrades  or  changes.  (PEO 
IEW&S,  2013,  p.  23) 

The  program  office  conducted  numerous  T&E  events  of  the  CREW  Duke  system 
prior  to  full  fielding.  Some  of  these  events  were  in  addition  to  the  requirement  of  fielding 
a  RA  with  the  desired  end  state  of  becoming  a  POR  in  the  future.  Each  CREW  Duke 
configuration  underwent  DT.  DT  evaluated  aspects  of  mission  equipment,  the  electrical 
power  source,  and  integration  into  a  prime  mover.  AEC  conducted  a  formal  Capabilities 
and  Limitations  (C&L)  Test  prior  to  the  initial  deployment,  which  employed  priority 
threats  in  a  representative  operational  environment  (PEO  IEW&S,  2012,  p.  1-6).  The 
AEC,  as  the  independent  developmental  and  operational  evaluator,  is  a  principal  T&E 
IPT  member  and  chairs  all  RAM  scoring  conferences.  AEC  supplied  documentation 
assessing  how  well  the  CREW-2  systems  met  the  CPD  developed  by  the  Combat 
Developer  (CBTDEV),  including  performance  validation  data,  FOA  feedback,  and 
provided  analysis  in  a  CLR.  AEC  monitors  key  tests  and  prepares  the  Abbreviated 
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Operational  Evaluation  Report  (OER-A)  and  Materiel  Release  Position  Memo  to  support 
the  FMR  (PEO  IEW&S,  2012,  p.  2-2). 

As  described  in  the  CREW-2  Duke  TEMP  (authored  by  PEO  IEW&S),  the 
following  list  documents  the  testing  conducted  prior  to  being  designated  a  POR  to 
include: 

•  System  Testing:  OEM  implemented  CREW  system  test  program  to 
ensure  that  compliant  Duke  systems  of  the  highest  quality  were  delivered 
to  the  Government.  This  included  a  thorough  range  of  First  Article  Tests 
(FAT),  to  include  a  Reliability  Demonstration  for  each  phase,  and  a 
follow-on  AEC-evaluated  test  at  Yuma  Proving  Ground  (YPG),  AZ.  The 
tests  conducted  by  AEC  utilized  operationally-placed  threats  (2012,  p.  1- 
6). 

•  FAT:  conducted  by  the  OEM  at  the  contractor’s  facilities  and  witnessed 
by  the  government.  The  following  tests  were  conducted  for  each  CREW 
Duke  version:  high/low  temperature,  solar  loading,  shock/vibration,  sand 
and  dust  intrusion,  rain/blowing  rain  intrusion,  ice/  freezing  rain  intrusion, 
humidity/salt  fog/fungus  intrusion,  electromagnetic  interference  (EMI), 
electromagnetic  compatibility  (EMC),  and  reliability  demonstration  (2012, 
pp.  1-6  -  1-7). 

•  Software  Functional  Qualification  Test  (SFQT):  to  stress  the  software 
and  ensure  a  stable  software  baseline  acceptance  (2012,  p.  1-7). 

•  Reliability  demonstration:  OEM  conducted  Reliability  Demonstrations 
for  each  version  to  verify  that  the  Duke  systems  met  the  minimum 
reliability  requirements  with  a  confidence  level  that  equals  or  exceeds  the 
requirement  specified  in  the  CPD  (2012,  p.  1-7). 

•  Safety  and  Health  Hazards:  initiated  by  the  OEM  and  evaluated  by  the 
Communication-Electronics  Command  (CECOM)  Safety  Office  and  the 
former  Developmental  Test  Command  (DTC)  covering  all  aspects  of  the 
Duke  system  to  ensure  all  hazards  related  to  installation,  transportation, 
operation,  maintenance,  and  storage  of  Duke  systems  were  eliminated  or 
minimized.  The  Program  office  also  required  the  contractor  to  develop  a 
Safety  Assessment  Report  (SAR)  and  complete  the  Health  Hazard 
Assessment  (HHA),  as  part  of  the  safety  release  and  safety  confirmation 
process  (certified  in  3Q  FY07  for  Duke  V2  and  2Q  FY09  for  Duke  V3) 
(2012,  p.  1-7). 

•  Training:  evaluation  was  conducted  to  determine  that  all  critical  tasks 
related  to  system  operation,  maintenance,  and  support  could  be  performed 
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in  the  specified  environment  by  soldiers  in  the  field,  with  minimal  training 
(2012,  p.  1-7). 

•  Transportability:  engineering  analysis  of  the  host  platforms 
incorporating  the  CREW  Duke  system  was  conducted  to  determine 
potential  transportation  problems  and  to  ensure  transportability 
requirements  were  met  (2012,  p.  1-7). 

•  Electromagnetic  Environmental  Effects  (E3):  conducted  during  EMI 
testing  during  FAT  and  simulated  RF  environments  operating  during  all 
field  tests  (2012,  p.  1-8). 

•  Reliability  and  Maintainability:  A  selected  number  of  Duke  pre- 
production  units  were  tested  in  accordance  with  the  Reliability  Test  Plan 
(RTP),  developed  by  the  OEM  (2012,  p.  1-8). 

•  Performance:  CREW  Duke  performance  was  evaluated  to  determine  if  it 
met  the  requirements  of  the  PBS  and  CPD  for  the  CREW-2  system(s). 
This  requirement  was  continuously  tested,  throughout  the  life  of  the 
system,  including  evaluated  tests  for  each  phase  and  continuously  repeated 
for  the  numerous  load  set  modifications  required  to  stay  relevant  against 
new  and  emerging  threats  during  all  phases  of  the  CREW  Duke  program. 
A  fully  developed  traceability  matrix  from  evaluation  domain  to  Critical 
Operational  Issue  (COI)  to  Measure  Of  Effectiveness  (MOE)  to  Measure 
Of  Performance  (MOP)  was  not  developed  until  later  in  the  program  and 
initial  performance  was  judged  strictly  against  KPPs  and  KSAs  identified 
in  the  requirements  documentation.  ATEC  issued  a  Duke  V3  CLR  in  3Q 
FY10  (2012,  p.  1-8). 

•  Mobility:  Dimensional,  weight,  performance  and  center-of-gravity  data 
were  measured  and  compared  with  the  constraints  of  the  host  vehicles  to 
demonstrate  the  Duke  system’s  ability  to  keep  up  with  the  supported  unit 
and  support  the  maneuver  force.  Mobility/Vibration  road  tests  were  also 
conducted  at  Aberdeen  Proving  Ground  (APG),  MD.  The  objective  of  this 
testing  was  to  verify  that  the  Duke  system  is  capable  of  withstanding 
transportability  stresses  in  the  field  and  operating  satisfactorily  during  and 
after  exposure  to  mobility  stress  and  vibration,  as  would  be  encountered  in 
vehicle  movement  over  rough  terrain  (2012,  p.  1-8). 

As  recognition  that  the  CREW  Duke  program  was  transitioning  from  RA  to  POR, 
program  documentation  was  structured  to  capture  all  previous  T&E  events  as  an  attempt 
to  highlight  any  critical  information  that  was  still  required.  Due  to  the  fact  that  the  CREW 
Duke  system  started  out  as  a  RA,  a  significant  portion  of  the  technical  review  and  audit 
processes  found  in  a  traditional  acquisition  program  was  abbreviated.  CREW  Duke 
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system  acquisitions  were  based  on  a  theater  ONS  and  supporting  urgent  requirements, 
which  mandated  fielding  to  both  OIF  and  OEF  as  quickly  as  possible.  Consequently,  PM 
CREW  analyses  performed  during  the  early  CREW  2.0  system  development  and 
acquisition  (i.e.,  production  and  fielding)  resulted  in  a  reduced  set  of  technical  data 
requirements.  To  complete  the  transition  to  a  POR,  the  Army  evaluator  identified  follow- 
on  T&E  events  that  were  required  after  fielding:  system  activity  monitor  test  and 
Logistics/Maintenance  Demonstration  (L/M  Demo)  (PEO  IEW&S,  2012,  p.  3-1). 

The  objective  of  the  system  activity  monitor  test  is  to  evaluate  the  self -monitoring 
capability  of  CREW  Duke  system  and  the  length  of  time  the  system  can  operate  on 
vehicle  power  (PEO  IEW&S,  2012,  p.  3-5).  The  L/M  Demo  was  conducted  at  CREW 
University  in  Edgewood,  MD,  during  the  Summer  of  2012,  with  a  nondestructive 
disassembly  and  re-assembly  of  a  production-representative  system  using  all  associated 
support  materiels.  This  event  provided  data  regarding  adequacy  of  maintenance  planning, 
System  Support  Package,  Life  Cycle  Support  Plan  (LCSP),  maintenance  documentation, 
and  training  (PEO  IEW&S,  2012,  pp.  3-4,  3-5-3-6). 

Test  limitations  were  also  identified.  “Not  all  threats  or  emplacements  were 
exercised  during  field  testing,  however  this  was  mitigated  by  focusing  on  the  highest 
priority  threats  as  determine  by  theater  and  the  Joint  IED  Defeat  Organization  (JIEDDO)” 
(PEO  IEW&S,  2012,  p.  3-4). 

Reliability  growth  was  not  a  factor  during  the  initial  design  and  fielding  of  the 
CREW  Duke  system.  It  was  initially  intended  to  be  a  RA  to  fill  an  immediate  need,  but 
its  long-term  importance  became  apparent  as  fielded  quantities  increased.  There  is 
insufficient  testing  scheduled  at  this  time  to  demonstrate  reliability  growth  and,  because 
production  of  the  system  is  complete,  little  value  would  be  realized  by  the  development 
of  a  reliability  growth  plan  unless  significant  reliability  problems  arise  (PEO  IEW&S, 
2012,  p.  3-5). 
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IV.  ANALYSIS  AND  FINDINGS 


A.  PRIMARY  RESEARCH  FINDINGS 

1.  Primary  Finding  1 — Differences  between  RA  and  POR  for  T&E 

The  differences  between  a  RA  and  a  POR  in  terms  of  T&E  can  be  boiled  down  to 
three  key  areas:  requirements,  purpose,  and  intended  use. 

The  first  key  difference  is  the  determination  of  requirements.  Requirements  may 
be  the  single  most  important  factor  when  embarking  on  a  T&E  effort.  They  determine 
what  to  test,  they  aid  in  deriving  how  the  test  should  be  designed,  and  what  test  the 
results  should  be  measured  against. 

POR  follow  a  specific  requirements  process  that,  once  complete,  identifies  a  very 
specific  materiel  solution.  This  requirement  is  born  in  one  of  several  strategy  documents 
that  the  joint  community  conducts  a  Capability  Based  Assessment  against.  The  remaining 
gaps  manifest  themselves  in  the  form  of  an  ICD.  The  ICD  identifies  the  requirement  in  a 
very  broad  manner,  allowing  the  acquisition  community  to  determine  ways  to  address  the 
identified  capabilities  gaps.  Meanwhile,  the  Capability  Developer  begins  work  on  a 
requirements  document  that  will  identify  KPPs  and  KSAs.  These  KPPs  and  KSAs  will 
determine  what  the  system  needs  to  do  in  operational  terms.  This  should  provide  the  test 
community  and  program  management  team  with  the  information  required  to  develop  a 
test  and  evaluation  strategy  to  support  the  acquisition  strategy. 

A  RA  program  is  executed  from  a  much  smaller  data  set  with  far  less  analysis. 
The  analysis  could  occur  at  the  tactical  level,  service  level,  or  at  the  COCOM  level.  This 
occurs  only  to  solve  an  existing  or  emerging  problem.  It  does  not  take  into  account  the 
long-term  implications  or  capability  gaps  outside  of  the  immediate  need.  This 
requirement  lacks  the  fidelity  of  a  KPP  and  lacks  the  uniformity  that  the  JCIDS  process 
provides.  ATEC  Regulation  73-1  address  this  problem  as  follows: 
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One  of  the  key  deficiencies  in  the  RAI  process  is  the  Joint  Use 
Operational  Needs  Statement  (JUONS),  Operational  Needs  Statement 
(ONS),  and  the  abbreviated  format  known  as  the  ten  liner.  These 
documents  vary  greatly  in  quality  and  for  the  most  part  do  not  express 
substantive  operational  requirements  and  associated  key  performance 
parameters  or  critical  technical  parameters.  As  such,  it  is  incumbent 
upon  the  AST  chair  to  use  all  means  to  bridge  this  requirements  gap 
and  produce  a  CLR  that  provides  stakeholders  with  sufficient 
information  so  that  readers  can  draw  operational  effectiveness  and 
suitability  conclusions  or  seek  additional  data.  It  is  critical  that  these 
customers  know  as  much  as  possible  about  the  equipment,  including 
the  associated  technical  and  operational  risks  inherent  in  its 
employment,  as  well  as  any  unknowns.  (ATEC,  2013) 

The  starting  point  of  T&E  is  the  requirement.  The  clarity  of  the  requirement 
ensures  testing  can  deliver  quantifiable  results  that  can  be  used  to  measure  suitability, 
survivability,  and  effectiveness.  POR  have  a  clearer  T&E  path  based  on  sound 
requirements,  where  RAs  requirements  are  less  clear  and  require  interpretation  by  the  test 
community  and  the  program  management  team. 

The  second  key  difference  between  POR  and  RA  is  the  purpose  of  the  testing.  For 
a  POR,  testing  is  conducted  to  determine  if  KPPs  and  KSAs  are  met.  In  RA,  an 
evaluation  takes  place  to  determine  the  capabilities  and  limitation  of  the  system.  In  the 
case  of  POR,  if  a  test  event  results  in  a  threshold  requirement  not  being  met,  the  program 
could  face  cancelation.  This  means  the  PM  must  attempt  to  address  shortcomings  and 
deficiencies.  This  test-fix-test  approach  allows  the  PM  to  achieve  KPPs  without 
compromising  the  program.  This  information  is  used  to  support  an  acquisition  decisions 
at  key  junctures.  Each  phase  of  the  process  has  key  tests  that  are  performed  to  support 
key  decisions. 

On  the  other  hand,  the  RA  goes  through  limited  DT  and  an  OA  that  evaluates  the 
system  against  stated  requirements,  and  against  each  of  the  DOTMLPF-P  domains.  This 
evaluation  is  designed  to  inform  stakeholders  of  the  performance  capabilities  and 
limitations  of  the  proposed  solution.  It  does  not  support  an  acquisition  decision  because 
(normally)  the  materiel  solution  has  already  been  procured.  The  information  from  this 
evaluation  will  be  used  by  the  commander  to  support  user  acceptance,  and  by  the  MDA 
to  support  an  UMR. 
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The  final  difference  between  a  POR  and  RA  is  the  intended  purpose.  A  RA  is 
developed  for  a  very  specific  capability  in  a  very  specific  environment,  and  for  a  specific 
period  of  time.  When  a  UMR  is  granted,  it  is  only  for  the  specific  use  identified.  The 
Safety  Confirmation  covers  the  materiel  for  a  limited  use  in  a  specified  environment.  If  a 
UMR  is  granted  for  use  in  one  Area  of  Operations  (AOR)  for  a  commander,  it  cannot  be 
transferred  to  another  commander  in  another  AOR  without  updating  the  UMR.  The 
Safety  Confirmation  is  very  specific  in  this  regard.  An  FMR  focuses  on  the  system’s  full 
safety  and  hazard  elimination/acceptance,  suitability,  and  supportability.  Above  all,  the 
involvement  of  ATEC  to  carry  out  the  activities  identified  in  the  TEMP  requires  an 
extensive  T&E  program  (increasing  program  cost  and  schedule)  in  order  to  demonstrate 
effectiveness,  survivability,  Manpower  and  Personnel  Integration  (MANPRINT) 
compliance,  reliability,  supportability,  and  interoperability.  Knowing  these  T&E  goals 
early  allows  a  PM  to  better  focus  and  to  conduct  systems  engineering  efforts  that  result  in 
the  best  solution  for  the  Soldier.  For  a  POR,  an  FMR  is  signed  as  part  of  the  acquisition 
process  leading  to  a  FRP  decision  and,  once  fielded  to  the  Services,  it  can  be  employed 
anywhere  the  Services  need  it.  It  means  the  system  has  been  tested  for  suitability  in  all 
AORs  and  can  be  employed  by  all  commanders.  If  a  unit  requires  this  capability,  it  can 
request  this  equipment  through  command  channels.  The  FMR,  unlike  the  UMR,  is  good 
for  the  life  of  the  equipment  unless  updated  by  the  MDA. 

The  key  differences  between  POR  and  RA  when  it  comes  to  T&E  are  the 
requirements  generation,  the  intended  purpose  of  the  T&E  activities,  and  the  purpose  of 
the  equipment  being  used. 

2.  Primary  Finding  2 — Effect  of  Abbreviated  T&E  Requirements 

Our  research  has  indicated  that  there  are  two  primary  effects  of  abbreviated  T&E 
requirements.  The  first  is  time  required  for  fielding,  and  the  second  primary  effect  is  risk. 

In  an  RA  environment,  time  is  the  most  critical  element.  Time  represent  lives  lost, 
or  possibly  mission  failure.  The  luxury  of  time  may  not  be  available  to  a  PM  responsible 
for  an  RA.  In  these  cases,  the  PM  and  the  Test  community  are  required  to  provide  a 
solution,  ensure  the  solution  is  safe,  and  evaluate  its  capabilities  and  limitations.  This 
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may  provide  materiel  solutions  to  warfighters  with  a  less  than  ideal  capability  but  in  time 
to  affect  the  immediate  capability  gap.  This  exposes  the  user  to  risk  and  can  be  a  source 
of  unseen  cost  increases.  In  the  case  of  the  DR  SKO,  some  of  the  RA  systems  remained 
in  the  field  for  over  a  decade  until  the  POR  systems  replaced  them.  The  effect  of 
abbreviated  T&E  is  the  ability  to  provide  stop  gap  until  the  capability  is  no  longer  needed 
or  until  the  long-term  solution  can  be  put  in  place.  Given  the  service  life  of  equipment 
and  its  lifetime  enormous  cost,  T&E  ensures  that  our  investments  are  protected.  In  most 
cases,  RAs  do  not  have  that  luxury.  The  DR  SKO  and  the  CREW  Duke  were  needed  to 
fulfill  immediate  capability  gaps.  The  threat  of  WMDs  and  IEDs  pose  significant 
challenges  to  commanders.  Abbreviated  T&E  for  the  CREW  Duke  RA  allowed  the 
capability  to  be  fielded  rapidly  and  save  lives  immediately  by  providing  commanders  a 
capability  until  the  POR  elements  could  catch  up.  The  DR  SKO  pre -program  versions 
provided  commanders  with  an  upfront  ability  to  detect  the  presence  of  a  variety  of 
harmful  materiels  that  had  previously  gone  undetected.  The  rapid  nature  of  the  T&E 
conducted  provided  the  warfighter  with  a  capability  that  reduced  gaps  and  bought 
commanders  the  necessary  time  required  for  the  POR  to  become  established. 

The  second  element  is  risk.  When  the  T&E  process  is  compressed  to  meet 
shortened  timelines,  many  tests  that  normally  occur  are  omitted.  The  pre  POR  versions  of 
DR  SKO  did  not  conduct  a  significant  amount  of  DT.  In  the  case  of  RA,  the  services  are 
left  to  manage  any  residual  risk.  The  assumed  risk  is  much  broader  than  injury  or 
inability  to  completely  fill  a  capability  gap.  In  one  example,  the  risk  comes  in  the  form  of 
unmet  requirements.  Due  to  the  need  for  a  compressed  timeline,  the  PM  for  the  DR  SKO 
RA  program  used  manufacturer  claims  in  assist  in  procurement  decisions.  The  DR  SKO 
POR,  on  the  other  hand,  used  DT  to  verify  those  manufacturer  claims  and  then 
disqualified  those  that  did  not  meet  the  defined  requirement. 

Some  of  the  risk  also  occurs  in  the  form  of  increased  cost  of  employing  RA.  One 
increase  in  cost  of  RA  can  be  attributed  incomplete  RAM  data  collected.  The  CLR 
provides  some  RAM  information,  but  it  is  mostly  limited  to  observations  during  the  OAs. 
In  this  case,  the  claims  of  the  manufacture  were  also  used  for  the  DR  SKO  RA  program. 
In  contrast,  the  DR  SKO  POR  used  the  Army  Materiel  Systems  Analysis  Activity 
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Scorecard  (JPM  CA,  December  2012)  to  guide  the  program  through  the  process  of 
collecting  RAM  data.  The  POR  evaluated  RAM  factors  over  515  mission  hours  (JPM 
CA,  December  2012).  The  RA  simply  lacked  the  time  required  to  test  to  this  level  of 
fidelity.  The  lack  of  RAM  data  provided  an  incomplete  picture  for  the  service.  Without 
complete  RAM  data,  the  mission  suitability  of  the  RA  is  questionable. 

Additionally,  the  Service’s  logistics  systems  are  not  set  up  to  support  RA  in  the 
same  manner  they  support  POR.  RA  require  their  own  support  mechanism  that  may 
require  a  complete  Contractor  Logistic  Support  (CLS)  package.  The  execution  of 
maintenance  is  moved  out  of  the  commander’s  organic  prevue  and  into  the  hands  of  CLS 
maintainers.  This  CLS  may  require  additional  outside  personnel  that  require  additional 
logistics  support,  such  as  security,  facilitates,  and  berthing  placing  additional  strain  on 
units.  CLS  support  usually  requires  a  higher  funding  level  from  the  PM  as  well.  The  DR 
SKO  POR  conducted  a  Logistics  Demonstration  during  the  first  quarter  of  Fiscal  Year 
2013  that  documented  each  of  the  12  Integrated  Product  Support  Elements.  Once 
complete,  the  PM  team  had  the  information  required  to  determine  the  suitability  of  the 
POR.  This  information  was  not  available  to  the  RA  program  and  created  the  need  for 
CLS. 

The  final  element  of  risk  that  is  not  accounted  for  in  RA  is  the  cost  of  system- 
level  replacement.  PORs  have  a  known  shelf-  life  accounted  for  in  POM  cycles.  The  DR 
SKO  CDD  established  clear  Ownership  Cost  metrics  with  a  threshold  of  $564  million 
and  an  objective  of  $508  million  (JPM  CA,  December  2012).  The  life  cycle  of  an  RA 
may  be  unknown  and  may  require  upgrading  and  refresh  multiple  times  throughout  its 
mission  life.  The  main  concern  of  an  RA  is  to  ensure  a  capability  reaches  the  warfighter 
with  ownership  costs  becoming  a  secondary  factor. 

A  subset  within  the  risk  element  relates  to  when  a  system  successfully  fills  a 

requirement  and  is  identified  for  long-term  use  through  either  the  CDRT  process  or  the 

NS-E  AROC  process.  The  system  can  be  identified  as  an  APC  or  at  least  the  recipient  of 

further  funding.  If  classified  as  an  APC  and  required  to  enter  the  JCIDS  process,  the 

program  office  will  need  to  generate  all  of  the  mandatory  acquisition  documentation  that 

may  have  been  not  required  under  a  RA,  including  the  T&E  activities.  The  risk  is  that  the 
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T&E  activities  may  reveal  system  shortcomings  due  to  the  limited  amount  of  time 
available  to  conduct  systems  engineering  design  and  analysis  on  the  original  system. 
While  the  capabilities  were  provided  to  the  Soldier  that  needed  it  in  the  short  term,  a  full 
T&E  program  may  reveal  complications  in  sustaining  it  in  the  Army  inventory  for  the 
long  term. 

The  effects  of  an  abbreviated  T&E  process  allow  the  commander  to  gain  time  in 
the  filling  of  a  capability  gap,  but  also  adds  elements  of  risk  that  would  not  be  present  in 
standard  POR  with  complete  testing  protocols.  The  services  must  weigh  these  risks 
during  the  requirements  determination  and  validation  process  used  to  approve  RI. 

B.  SECONDARY  RESEARCH  FINDINGS 

1.  Secondary  Finding  1 — Benefits  and  Drawbacks  for  CREW 

IEDs  became  a  very  significant  issue  during  OEF  and  OND,  from  both  an 
operational  stand  point  as  well  reducing  national  moral.  The  U.S.  and  Coalition  ground 
forces  were  caught  off  guard  by  the  widespread  and  successful  use  of  IEDs  (specifically 
RCIEDs)  and  needed  to  compensate  for  the  asymmetric  warfare  they  encountered.  The 
Army  required  a  solution  that  could  address  the  wide  spectrum  of  threats  and,  needed  it 
quickly.  The  CREW  Duke  system  was  the  solution  to  that  gap. 

The  CREW  Duke  system  was  specified  to  meet  performance  against  a  set  of 
known  threats  through  a  PBS  as  opposed  to  defining  detailed  hardware  and  software 
requirements.  This  method  allowed  vendors  to  propose  solutions  they  had  in  a 
production-ready  state.  Instead  of  having  to  take  the  time  (that  was  not  available)  and 
additional  resources  to  develop  a  solution  from  scratch,  contractors  were  able  to  propose 
solutions  that  met  the  performance  requirements  without  having  to  adhere  to  strict 
government  guidelines.  For  example,  the  incorporation  of  a  reliability  growth  plan  was 
not  a  required  factor;  neither  was  the  requirement  to  utilize  a  modular  design  to  increase 
the  ease  of  upgrades  in  the  future.  In  addition,  the  associated  software  and  firmware  was 
not  required  to  be  written  to  allow  for  ease  of  modification  in  the  future.  This  allowed  for 
fielding  of  a  large  number  of  CREW  Duke  systems  over  several  years  throughout  OEF 
and  OND  to  address  the  RCIED  threat.  From  a  T&E  perspective,  only  the  final  system 
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performance  against  the  defined  set  of  threats  was  evaluated.  Again,  this  allowed  for  a 
quicker  solution  to  be  fielded  against  an  immediate  gap,  but  did  not  allow  the  Army  to 
have  full  visibility  on  software  inefficiencies,  all  possible  compatibility  risks,  or 
performance  limitations  for  future  RCIED  threats. 

The  CREW  Duke  program  did  benefit  from  the  high  visibility  (both  in  the  media 
and  within  the  DOD)  due  to  the  cost  in  lives  the  RCIEDs  were  taking.  This  also  created  a 
demand  for  many  working  groups  and  organizations  (such  as  JIEDDO)  that  were 
established  up  to  combat  the  IED  problem.  CREW  Duke  program  benefited  from  this 
visibility  since  multiple  avenue  to  secure  funding  for  un-pro grammed  costs.  For  example, 
of  the  19  data  sources  used  to  establish  CREW  Duke’s  ability  to  meet  suitability 
requirements,  17  of  them  were  generated  from  the  L/M  Demo  (PEO  IEW&S,  2012,  pp. 
D5  -  D9).  The  L/M  Demo  occurred  in  the  summer  of  2012,  after  fielding  and  designation 
as  a  POR  (PEO  IEW&S,  2012,  p.  4-1).  Many  of  these  unplanned  costs  resulted  from 
developing  and  adapting  the  supportability  strategy  in  real  time.  As  the  requirement  for 
more  systems  grew  and  as  the  posture  on  the  ground  in  theater  constantly  shifted,  the 
expected  lifetime  of  the  system  grew. 

As  noted  in  the  CREW  Duke  TEMP,  the  L/M  Demo  was  taking  place  after  full 
fielding  of  the  system  had  already  been  achieved.  Several  system  considerations  affecting 
the  long-term  life  cycle  of  the  CREW  Duke  system  (e.g.,  laptop  availability  for  software 
updates  or  the  correct  levels  spare  parts)  were  not  part  of  the  system-level  evaluation 
earlier  in  the  program.  For  example,  laptops  were  not  part  of  the  system  description  (PEO 
IEW&S,  2013,  pp.  14-19)  so  the  continued  success  of  the  system  relies  on  the 
availability  of  hardware  outside  the  purview  of  the  CREW  Duke  program.  Also,  the 
OEM  reliability  demonstration  was  able  to  show  the  system  level  reliability  of  the  CREW 
Duke  (PEO  IEW&S,  2012,  p.  1-7)  but  did  not  allow  for  identification  of  sub-system 
components  that  could  lower  system  reliability.  The  total  life  cycle  costs  were  not 
evaluated  early  in  the  program  because  speed  of  fielding  was  the  most  important  priority 
in  having  a  successful  program.  While  success  was  achieved,  conducting  the  L/M  Demo 
earlier  would  have  allowed  for  better  planning  for  the  total  life  cycle  cost  in  preparation 
for  transitioning  to  a  POR. 
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Additionally,  the  Data  Source  Matrix  identifying  the  requirements  traceability 
from  COI  to  MOE/MOP  to  KPP/KSA  was  not  complete  until  the  publishing  of  the 
CREW  Duke  TEMP  in  June  2012.  The  evaluation  framework  and  dendritic  were  defined 
in  the  TEMP  as  activities  not  yet  achieved  in  support  of  the  CREW  Duke  FMR  decision 
(PEO  IEW&S,  2012,  p.  3-1).  Since  performance  against  the  RCIED  threat  was  the  main 
criteria  for  successful  and  quick  fielding,  this  did  not  allow  the  Army  the  time  to  fully 
evaluate  the  requirements  traceability  and  only  required  the  L/M  Demo  to  come  later  in 
the  program.  While  many  lessons  learned  have  already  been  applied  to  the  maintenance 
and  supportability  activities  of  CREW  Duke,  the  transition  to  a  POR  may  have  required 
less  follow-up  T&E  effort  if  the  traceability  was  conducted  earlier. 

Overall,  the  quick  fielding  of  the  CREW  Duke  program  was  a  great  benefit  to  the 
Army  because  it  was  able  to  effectively  combat  the  RCIED  threat.  The  transition  to  a 
POR  does  require  following  up  with  several  T&E  activities  to  complete  the  forward 
planning  for  the  entire  life  cycle  of  the  system  and  fulfill  all  the  requirements  for  FMR. 
Specifically  for  CREW  Duke,  this  included  the  L/M  Demo  to  evaluate  the  “logistical 
capability  required  to  sustain  the  system”  as  well  as  the  system  activity  monitor  test  used 
to  evaluate  the  system’s  self-monitoring  capability  (PEO  IEW&S,  2012,  p.  3-5). 

2.  Secondary  Finding  2 — Benefits/Drawbacks  for  DR  SKO 

The  DR  SKO  POR  got  a  head  start  in  the  acquisition  process  because  of  the  work 
done  during  the  RA  effort.  The  TIC  Protection  and  Detection  Equipment  (PDE)  ONS 
opened  the  door  by  fielding  16  systems  in  2004.  An  additional  40  systems  were  added  in 
2010  after  a  JUONS  was  submitted.  The  Joint  Nuclear  Biological  Chemical  Radiological 
System  II  (JNBCRS  II)  (sometimes  referred  to  as  the  JUONS  II  kit)  replaced  the  TIC 
PDE  ONS  systems.  The  JNBCRS  II  remained  in  the  field  until  they  were  replaced  by  DR 
SKO  POR  systems.  This  provided  the  acquisition  community,  as  well  as  the  user 
community,  with  six  years  of  data  that  could  be  evaluated  and  analyzed  and  determine  if 
the  systems  were  filling  capability  gaps.  Throughout  this  period,  the  PM  did  not  remain 
idle. 


50 


As  with  any  RA,  the  PM  was  required  to  obtain  a  CLR  and  a  Safety 
Confirmation.  This  allowed  the  PM  to  conduct  required  system  level  safety  testing.  An 
OA  was  conducted  that  allowed  users  to  use  the  system  in  controlled  conditions  where 
measurable  data  was  collected.  Several  instruments  were  tested  at  Dugway  Proving 
Grounds  using  Chemical  Warfare  Agents  (CWA)  and  TICs.  The  intent  was  to  secure  a 
CLR  and  Safety  Confirmation,  but  as  a  byproduct,  volumes  of  data  were  collected  to 
support  the  future  POR.  They  further  developed  the  RA  by  containerizing  the  system  in 
Quad  Con  transportation  containers.  This  design  was  proven  successful  at  another  OT 
conducted  with  users  at  Ft.  Hood,  Texas.  During  this  same  period,  the  PM  funded  ATEC 
to  evaluate  the  RA  system  against  the  emerging  requirements  being  codified  in  the  CDD. 

The  DR  SKO  program  clearly  benefited  from  its  RA  predecessors  in  several 
areas.  Due  to  the  RA  work,  the  need  to  conduct  and  Analysis  of  Alternatives  (AOA)  was 
alleviated.  An  AOA  is  designed  to  ensure  that  all  other  materiel  options  have  been 
considered  and  cannot  fulfill  the  existing  capability  gap  effectively  and  efficiently.  This 
is  a  lengthy  process  by  design  that  could  take  one  to  two  years  from  start  to  finish.  In  this 
case,  the  MDD  determined  that  the  Analysis  of  Materiel  Solution  met  the  requirements  of 
an  AOA. 

Additional  benefits  were  realized  from  previous  data  mining.  The  DR  SKO 
TEMP  lays  out  four  clear  efficiencies  gained  from  the  work  completed  during  the  POR: 

1.  Reduce  the  MOT  test  locations  from  three  separate  locations  to  two 
separate  locations  (land  and  maritime)  (JPM  CA,  December  2012). 

2.  Reduce  the  MOT  schedule  by  completing  a  standalone  RAM  Test  during 
the  PQT  (a  MOT  test  schedule  reduction  of  300  hours)  (JPM  CA, 
December  2012). 

3.  Reduce  the  Log  Demo  from  4  participating  Services  to  2  (USA  and 
USMC)  (JPM  CA,  December  2012). 

4.  Reduce  the  scope  of  Vapor  Sampler  testing  from  six  TICs  to  one  TIC 
(JPM  CA,  December  2012). 

The  benefits  from  the  RA  to  the  DR  SKO  POR  are  evident.  What  is  not  as  clear 
are  any  drawbacks  that  were  present.  During  the  course  of  research  for  this  project,  it 
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appeared  that  the  DR  SKO,  through  an  excellent  program  acquisition  strategy,  was  a 
model  for  how  to  transition  a  RA  to  a  POR. 

The  major  concern  with  this  method  is  how  the  POR  requirements  were 
developed.  It  is  impossible  to  determine  from  this  perspective,  but  the  question  should  be 
asked:  “did  the  RA  drive  the  requirement  or  did  the  capability  gap  drive  the 
requirement?” 

Any  time  a  program  is  this  successful,  it  is  easy  for  the  requirement  developers  to 
default  to  standards  that  exist  regardless  of  their  relevancy.  In  the  case  of  the  DR  SKO, 
the  requirement  was  written  to  a  civilian  standard,  not  to  an  operational  requirement. 
While  the  missions  of  civilian  Frist  Responders  and  DOD  CBRN  Reconnaissance 
Personnel  have  similarities,  they  are  inherently  different.  The  dynamics  of  combat  and 
the  countless  variables  encountered  are  not  accounted  for  in  the  civilian  standards  used. 
In  fairness,  development  of  DOD  unique  standards  is  very  difficult,  costly,  and  time 
consuming.  This  may  have  been  a  tradeoff  the  combat  developers  made  early. 

C.  SUMMARY 

There  are  clear  differences  between  the  test  and  evaluation  of  a  RA  and  a  POR. 
The  rigor  that  is  required  to  support  a  POR  is  not  present  in  a  RA.  This  is  done  to  get 
something  that  will  immediately  impact  a  mission  or  saves  lives  to  the  field  as  soon  as 
possible.  In  essence,  the  RA  framework  shifts  risk  from  the  program  manager  to  the 
commander.  We  evaluated  the  CREW  Duke  and  the  DR  SKO  programs,  which  are  both 
models  of  how  programs  can  be  transitioned  from  a  RA  to  a  POR.  In  both  cases, 
capturing  data  during  RA  activities  that  will  support  the  POR  is  key  in  shortening  the 
timeline  of  the  POR.  Program  Managers  should  remain  cognizant  of  transition 
opportunities  when  developing  RA. 
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V.  CONCLUSIONS,  RECOMMENDATIONS,  SUMMARY  AND 
AREAS  FOR  FURTHER  RESEARCH 

A.  CONCLUSIONS  AND  RECOMMENDATIONS 

We  began  this  research  by  asking  two  questions:  Did  the  abbreviated  T&E 
requirements  in  a  RA  environment  dilute  the  important  role  T&E  should  play  in  an 
acquisition  program?  What  are  the  long-term  effects  on  programs  from  the  manner  that 
T&E  is  conducted  to  meet  RA  timelines? 

The  results  of  our  research  exposed  a  very  different  result  to  the  first  research 
question  than  expected.  We  found  that  the  role  T&E  plays  in  RA  is  not  diluted  at  all  and 
is  in  fact  extremely  important.  An  RA,  by  design,  requires  a  shortened  timeline  to  deliver 
equipment  to  warfighters  in  order  to  saves  lives  and  ensure  mission  success.  T&E  in  this 
case  serves  a  very  important  role  during  the  production  of  a  Capability  and  Limitation 
Report  and  a  Safety  Confirmation.  The  PM,  working  with  the  test  community,  must 
determine  what  testing  will  take  place  and  in  what  form.  For  RA,  the  PM  can  conduct  all 
types  of  testing  to  ensure  the  materiel  solution  meets  the  requirements  stated.  The  user 
trusting  the  PM  and  T&E  process  is  asked  to  accept  the  capability  and  its  associated  risks. 

In  some  cases,  these  RA  programs  transition  into  POR.  The  Army  uses  the  CDRT 
process  to  transition  RA  to  POR.  CDRT  allows  for  the  entrance  of  capabilities  into  the 
later  stages  of  JCIDS  process.  Regardless  of  where  the  capability  enters  the  process, 
some  or  all  of  the  test  data  may  be  usable  for  the  POR. 

The  long-term  effects  of  T&E  activities  on  RA  programs  can  be  measured  in  both 
cost  and  schedule.  The  challenge  for  the  Services,  PM,  and  test  community  is  to  conduct 
RA  testing  in  a  manner  that  allows  for  the  data  to  be  usable  for  a  future  POR.  This  may 
require  additional  resources  initially,  but  could  be  a  precursor  for  future  savings.  Many 
lessons  were  learned  from  the  explosion  of  RA  fielding  activities  in  Iraq  and  Afghanistan. 
The  Army  Acquisition  community  was  forced  to  be  agile  in  a  time-constrained 
environment  to  meet  the  threat  of  asymmetric  warfare.  The  capability  and  process  to 
move  this  quickly  cannot  be  forgotten  at  the  risk  of  not  being  able  to  meet  the  next 
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unforeseen  threat.  A  process  similar  to  CDRT  or  the  NSE-AROC  is  recommended  to 
become  part  of  the  recognized  acquisition  process  in  order  to  codify  a  path  for 
capabilities  to  move  into  the  Army  inventory.  These  capabilities  can  be  recognized  as 
important  but  may  have  come  from  outside  the  traditional  development  and  acquisition 
process  (such  as  a  COTS  device  or  a  repurposed  piece  of  equipment).  A  critical  piece  of  a 
permanent,  successful  CDRT-like  process  would  be  identifying  a  required  set  of  T&E 
activities.  This  needs  to  be  done  prior  to  consideration  for  moving  a  capability  into  the 
Army  inventory  so  that  precious,  quick-reaction  time  is  not  lost  when  a  capability  needs 
to  move  to  the  field  immediately. 

Additionally,  it  is  recommended  that  the  Army  Research  and  Development 
(R&D)  community  proceed  with  integration  of  new  technologies  at  the  system  level  for 
near  term  deployment  while  keeping  in  mind  the  potential  effects  on  total  life  cycle  costs. 
When  a  technology  matures  past  a  lab  based  prototype,  testing  efforts  beyond  the 
baseline  performance  validation  are  required  to  prove  out  the  measures  of  suitability  and 
effectiveness.  The  R&D  community  must  identify  a  transition  path  for  new  technology  to 
reduce  the  risk  of  moving  too  far  down  a  path  without  a  clearly  identified  end  state.  This 
work  up  front  will  place  new  capabilities  in  a  better  position  for  a  smoother  and  quicker 
transition  to  the  Warfighter,  without  the  need  to  back  track  to  T&E  events  that  were 
initially  determined  unnecessary. 

B.  AREAS  FOR  FURTHER  RESEARCH 

The  amount  of  time  RA  remain  in  the  field  until  they  are  retired,  or  transition  into 
POR  should  be  examined  by  the  Services  and  Congress.  The  shear  amount  of  JUONS 
and  ONS  that  have  been  approved  over  the  last  decade  and  remain  in  the  field 
demonstrate  a  need  to  understand  what  their  residual  effects  are  on  the  Warfighter  and  the 
Operational  Army.  A  study  of  how  this  has  impacted  the  Acquisition  process  should  also 
be  studied.  We  have  almost  a  generation  of  acquisition  professionals  that  were  present  in 
a  pre-war  environment.  The  impact  of  the  last  decade  may  reveal  some  key  lessons 
learned  to  better  support  the  process  in  the  future 
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APPENDIX.  MATERIEL  RELEASE  COMPARISON 


The  Materiel  Release  comparison  table  from  Army  Regulation  700-142 
(Department  of  the  Army,  2013,  pp.  18-20),  lists  the  requirements  for  materiel 
developers  to  follow  when  pursuing  materiel  releases. 


Full  materiel  release  requirements-safety 


Aspect  or  Character¬ 
istic 

Activity  or  Document 

FMR  Requirements 

Functional  Author 

ity 

Safe 

Hazards  are  identified 
and  eliminated  or  ac¬ 
cepted. 

1.  Supporting  safety  office  certification. 

2.  TSG  HHA  (see  AR  40-10  and  AR  602-2).1 

3.  AMC  EOD  supportability  statement  (see  AR 
75-1 5).2 

4.  Environmental  statement  (see  AR  200-1 ,  32 
CFR  651).3 

5.  Airworthiness  statement  (see  AR  70-62).4 

6.  SSRA  for  residual  hazards  (see  AR  385-10). 

7.  ATEC  (Army  Evaluation  Center)  safety  confir¬ 
mation  (see  AR  385-10). 

8.  Surface  or  weapon  danger  zone  (see  AR 
385-63). 

9.  Final  hazard  classification  (see  49  CFR  173 
and  TB  700-2). 

10.  NRC  license  (see  10  CFR  chap  1). 

1 1 .  Army  Fuze  Safety  Review  Board  Certification 
(see  AR  385-10). 

12.  Energetic  materials  qualification  (see  AOP-7 
edition  2  revision  1 ). 

13.  Ignition  System  Safety  Review  Board  Certifi¬ 
cation  (see  MIL-STD-1901  and  Standardized 
Agreement  (STANAG)  4368). 

14.  Safety  review  of  technical  manuals  (TM)  (see 
AR  25-30). 

15.  Results  of  safety  inspections  and  analyses. 

16.  Software  safety  statement. 

System  safety  aspects  have  been 
reviewed  and  verified  by  the  suppor¬ 
ting  safety  office. 

-All  known  safety  hazards  have 
been  eliminated  or  accepted  through 
the  SSRA  process  in  accordance 
with  AR  385-10. 

-All  statutory  requirements  are  met. 
-Applicable  regulatory  requirements 
are  met. 

-All  environmental  impacts  have 
been  identified,  mitigated  if  possible, 
and  documented  in  accordance  with 
the  National  Environmental  Policy 
Act  and  32  CFR  651. 

Safety  office 

Notes: 

1  The  HHA  report  is  provided  by  the  PHC  on  behalf  of  TSG. 

2  Determine  EOD  statement  applicability  using  DA  Pam  700-142.  The  EOD  statement  will  certify  that  validated  and  verified  render  safe  and  disposal  proce¬ 
dures,  tools  and  equipment,  and  training  aids  are  fielded  to  Army  EOD  units  and  EOD  schools  at  least  30  days  prior  to  MR  and  that  the  new  materiel  is  fully 
supportable  by  EOD  units.  It  will  also  certify  that  the  EOD  technical  manuals  have  been  approved  by  the  Military  Technical  Acceptance  Board  at  least  30 
days  prior  to  MR  (see  AR  75-15  to  determine  the  MATDEVs  responsibility  for  EOD  supportability  compliance  during  the  development  of  the  new  materiel). 

3  The  environmental  statement  must  certify  that  the  requirements  of  AR  200-1  and  32  CFR  651  have  been  met. 

4  If  a  statement  of  airworthiness  qualification  is  not  yet  available,  a  FMR  and  subsequent  FRP  decision  may  be  approved  providing  the  request  for  system 
airworthiness  has  been  submitted  in  accordance  with  AR  70-62  and  there  are  no  known  issues  that  would  prevent  issuing  the  applicable  airworthiness  doc¬ 
uments. 
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Full  materiel  release  requirements-supportability 


Aspect  or 
Characteristic 

Activity  or  Document 

FMR  Requirements 

Functional 

Authority 

Supportable 

Integrated  prod¬ 
uct  support 
(IPS)  elements 
(see 

AR  700-127) 

26.  Supportability  certification  - 
will  address  support  materiel 
(COEI  and  ASIOE),  end  item 
and  software  (see  AR  700-127). 

27.  USATA  supportability  state¬ 
ment  on  TMDE  or  ATE 

(see  AR  750-43). 3 

28.  TC  designation. 

29.  SDDC  TEA  transportability 
statement  (see  AR  70-47).4 

30.  Army  logistician  assessment 
(see  AR  700-1 27).5 

31.  Supporting  statements  for 
COEI  and  ASIOE. 

32.  Software  supportability 
statement  (normally  provided  by 
the  LCMC  SEC). 

-Key  LCSP  performance  aspects  have  been  achieved  as 
determined  by  the  functional  authorities.2 
-Maintenance  planning  has  been  accomplished  and  coordi¬ 
nated.  Army  preference  is  in  accordance  with  AR  750-1. 
-Manpower  and  personnel  requirements  to  operate  and 
maintain  the  system  have  been  identified  and  documented. 
-Adequate  supply  support  for  fielding  and  sustainment  of 
units  (interim  contract  support,  performance-based  logistics, 
organic)  has  been  established. 

-Support  equipment  is  identified  and  documented  at  the  ap¬ 
propriate  organization;  TMDE  supportability  has  been  ad¬ 
dressed;  footprint  Is  minimized. 

-Technical  data  rights  of  use  are  established. 

-TM  and  interactive  electronic  technical  manual  verification 
has  been  completed  by  the  Government. 

-Training  and  training  support  to  include  TADSS  and  am¬ 
munition  requirements  for  training  have  been  identified, 
developed,  and  documented;  training  is  available  for  all 
GCs  and  maintainers. 

-Maintenance  of  software  is  addressed  in  the  LCSP  soft¬ 
ware  development  plan  and  life  cycle  cost  estimate,  and 
hardware  for  mission-critical  systems  is  available  at  the  ap¬ 
propriate  organization. 

-Facilities  requirements  are  developed  and  documented 
(maintenance,  training  storage,  covered,  humidity  con¬ 
trolled,  and  so  on);  facilities  are  available. 

-Package,  handling,  storage,  and  transportation  system  is 
transportable  by  all  modes  as  specified  in  the  capability 
document. 

-Transportability  has  been  evaluated  by  SDDC  and  docu¬ 
mented  accordingly. 

-The  PM  has  programmed  funding  to  complete  LCSP  activi¬ 
ties  within  the  POM  (coordinate  with  the  DCS,  G-3/5/7 
(DAMO-TR)). 

-Ammunition  Stockpile  Reliability  Program  and  ammuni¬ 
tion  surveillance  procedures  are  in  place. 

Lead  LCMC  Integrated 
Logistics  Support  (ILS) 
center  or  ILS  director¬ 
ate 

Notes: 

1  The  supportability  certification  will  verify  that  key  aspects  of  the  LCSP  have  been  achieved;  detail  any  known  shortfalls  and  include  them  in  a  recom-  men¬ 
ded  get-well  plan.  A  system  receiving  a  FMR  that  has  ASIOE  at  less  than  FMR  must  get  acceptance  from  the  GC  prior  to  fielding. 

2  Systems  supported  by  planned  interim  contract  support  that  have  been  funded  and  have  a  transition  plan  for  a  longer  term  support  strategy  such  as  or¬ 
ganic  support  may  be  fully  materiel  released. 

3  The  TMDE  supportability  statement  is  not  required  if  TMDE  is  not  being  provided  to  the  operator  or  field  or  sustainment  maintenance  provider. 

4  The  SDDC  transportability  statement  is  not  required  if  a  system  is  found  to  be  a  transportability  nonproblem  item  in  accordance  with  AR  70-47. 

5  USAMMA  will  provide  an  Army  logistician  assessment,  system  effectiveness  assessment,  and  safety  statement  for  medical  materiel. 
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Full  materiel  release  requirements-suitability 


Aspect  or  Char¬ 
acteristic 

Activity  or  Document 

FMR  Requirements 

Functional  Authority 

Suitable 

-Effectiveness 

-Survivability 

-MANPRINT 

-Reliability 

-Supportability 

-Interoperability 

17.  ATEC  MR  position  memorandum.1 

18.  ATEC  OMAR  or  OER 
(see  10  USC  139,  DOTE).1 

-The  materiel  has  been  tested  and 
evaluated  in  accordance  with  the  ap¬ 
proved  Test  and  Evaluation  Master 
Plan.1 

-Established  requirements  of  the  capa¬ 
bilities  documents  have  been  met  or  a 
decision  has  been  made  by  the  CAP¬ 
DEV  to  accept  the  current  perform¬ 
ance;  requires  DCS,  G-3/5/7  endorse¬ 
ment. 

ATEC1 

19.  CIO/G-6  AIC  statement  based  upon  AIC  com¬ 
pletion  (see  AR  25-1 ). 

20.  Certificate  of  Networthiness  (see  AR  25-1 ). 

21.  DIACAP  certification  statement  (see  AR  25-2). 

-Software  to  include  embedded  soft¬ 
ware  within  platforms,  has  attained 

AIC. 

-Proper  certifications  for  networthiness 
and  DIACAP  are  attained. 

CIO/G-6 

22.  Communications  Security  Logistics  Activity 
(CSLA)  statement  for  COMSEC  supportability.2 

-COMSEC  supportability  and  availabil¬ 
ity  have  been  verified  by  CSLA. 

CSLA  for  Army- 
adopted  items 

23.  CAPDEV  training  assessment  (statement  of  ade¬ 
quacy  of  institutional  training  support)  (see  AR 
350-1). 

-Training  is  determined  adequate  per 
AR  350-1. 

CAPDEV 

24.  Software  suitability  statement  (normally  provided 
by  the  LCMC  SEC). 

25.  Quality,  reliability,  availability,  and  maintainability 
statement,  including  service  or  shelf  life  assurance, 
Ammunition  Stockpile  Reliability  Program  (see  AR 
702-6),  and  ammunition  surveillance  procedures 
(see  SB  742-1  ).3 

-Software  is  suitable. 

-Reliability,  availability,  and  main¬ 
tainability  requirements  have  been 
achieved. 

Lead  LCMC  system 
engineering  activity 

Notes: 

1  In  cases  where  U.S.  Army  Intelligence  and  Security  Command  or  U.S.  Special  Operations  Command  are  the  single  user,  they  may  perform  user  testing  in 
lieu  of  ATEC. 

2  The  CSLA  COMSEC  statement  is  not  required  when  the  materiel  does  not  contain  standalone  COMSEC  devices  and  supporting  materiels. 

3  In  some  cases  such  as  missiles,  the  functional  authority  may  waive  the  requirement  to  verify  reliability  with  statistical  confidence  because  of  limited  test 
assets  (normally  due  to  cost).  If  the  LCMC  quality  and  reliability  assessment  shows  that  there  is  only  a  low  risk  of  not  meeting  the  requirement(s),  then  the 
PM  may  establish  a  plan  to  verify  reliability,  availability,  and  maintainability  through  analysis  of  field  and  stockpile  test  data.  In  these  cases,  the  LCMC  quality 
or  reliability  assessment  will  show  that  a  rigorous  reliability,  availability,  and  maintainability  program  has  been  executed  and  present  the  qualitative  data  or 
analyses  that  provide  nonstatistical  confidence  in  meeting  the  requirement(s).  Such  a  program  is  outlined  in  SAE-JA1000  and  will  include  activities  such  as: 
failure  mode,  effects,  and  critically  analysis,  physics  of  failure  analyses  or  assessments  based  upon  analogous  or  previous  generation  systems  and  others 
(see  SAE-JA1 000-1). 
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Urgent  materiel  release  documentation  requirements 


Required  Documentation 

la.  User  requested 

-Joint  Urgent  ONS  (JUONS)1 
or 

-A  written  request  signed  by  a  general  offi¬ 
cer  or  civilian  equivalent  within  the  gaining 
unit's  chain  of  command 

-Prepared  by  Combatant  Command  (CCMD) 
and  coordinated  with  Joint  Staff. 

-Prepared  by  unit  commander,  endorsed  by 
chain  of  command,  and  submitted  to  the  DCS, 
G-3/5/7. 

-DCS,  G-3/5/7  ONS  validation  memoran¬ 
dum 

or 

-DCS,  G-3/5/7  directed  requirement  mem¬ 
orandum  (DAMO-CIC  or  DAMO-AOC) 

-Will  take  the  form  of  either  an  ONS  validation 
memo  or  message  traffic  prepared  by  DCS, 
G-3/5/7  (DAMO-CIC  or  DAMO-AOC)  commu¬ 
nicating  results  of  the  Army  Requirements  and 
Resourcing  Board.2’3 

1b.  HQDA  directed 

-DCS,  G-3/5/7  approved  capabilities  docu¬ 
ments  (for  example,  operational  require¬ 
ment  document,  ICD,  CDD,  or  CPD) 

And 

Capability  has  been  approved  by 

DCS,  G-3/5/7. 

-Pre-FRP  phase. 

-MR  activities  not  complete. 

-Capability  needed  urgently  by  field. 

-DCS,  G-3/5/7  directed  requirement  mem¬ 
orandum  (DAMO-CIC  or  DAMO-AOC) 

Will  take  the  form  of  either  a  directed  require¬ 
ment  memorandum  or  message  traffic  pre¬ 
pared  by  DCS,  G-3/5/7  (DAMO-CIC  or 
DAMO-AOC)  directing  the  fielding  of  equip¬ 
ment  that  has  not  been  materiel  released. 

2.  A  safety  and  health  data  sheet  with  a 
risk  assessment  for  the  materiel  system. 

Prepared  by  the  safety  office  summarizing  all 
known  safety  and  health  hazard  issues  and 
their  mitigation  plans.4- 5’ 6' 7 

3.  An  airworthiness  statement,  if  applica¬ 
ble. 

See  AR  70-52. 

4.  An  EOD  supportability  statement  from 
the  AMC  EOD  staff  officer,  if  applicable. 

Confirms  EOD  support  or  coverage  for  the 
UMR  action,  if  applicable. 

5.  PM  request  for  acceptance  from  the  GC 
or  requestor. 

This  statement  will  notify  the  GC  or  requestor 
of  all  known  equipment,  supportability,  and 
sustainment  issues.  This  statement  must  in¬ 
clude  all  known  environmental,  safety  and  oc¬ 
cupational  health  hazards,  operational  and 
support  limitations  to  include  interoperability 
limitations  and  use  restrictions.8 

6.  GC  acceptance  statement. 

The  GC  or  requestor’s  acceptance  statement, 
signed  by  a  general  officer  or  civilian  equiva¬ 
lent.6 

Notes: 

1  JUONS  do  not  require  DCS,  G-3/5/7  validation.  Validation  of  JUONS  will  normally  be  done  by  the  Joint  Staff  point  of  contact  listed  in  the  JUONS. 

2  The  Equipment  Common  Operating  Picture  database  and  directed  requirement  memo  will  include  the  system  or  materiel  quantity,  gaining  unit,  geographic 
location,  application,  and  destination's  point  of  contract  information  to  facilitate  the  UMR  action. 

3  DCS,  G-3/5/7  validation  is  not  required  if  the  unit  is  already  authorized  the  equipment  on  their  MTOE.  An  approved  DCS,  G-3/5/7  basis  of  issue  that  has 
not  been  applied  to  the  MTOE  will  also  sen/e  as  valid  authorization  and  not  require  a  separate  DCS,  G-3/5/7  validation. 

4  Review  the  safety  office  assessment  when  configuration  changes  are  made,  when  the  operational  mission  profile  is  changed,  when  an  operational  safety 
incident  occurs,  or  at  least  annually  to  reassess  any  safety  risk.  The  dates  of  reviews  and  reassessments  will  be  entered  and  tracked  in  the  MRTS. 

5  Coordinate  with  the  U  S.  Army  Public  Health  Command  (Health  Hazard  Assessment  Program)  for  inclusion  of  potential  health  hazard  information. 

6  Obtain  the  safety  confirmation  from  ATEC  (Army  Evaluation  Center). 

7  Prepare  and  coordinate  a  Safety  and  Health  Data  Sheet  for  acceptance  of  residual  safety  risks  by  the  GC  in  accordance  with  DA  Pam  385-16. 

8  Review  the  materiel  for  interoperability  certifications  such  as  AIC  and  DIACAP.  Complete  required  certifications  within  one  year  of  UMR  in  accordance  with 
CIO/G-6  guidance. 
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Training  materiel  release  requirements 


Aspect  or  Characteristic 

Activity  or  Document 

TMR  Requirements 

Functional  Authority 

Safety 

1.  Supporting  safety  office  certification. 

2.  TSG  HHA  (see  AR  40-10,  AR 
602-2). 1 

3.  AMC  EOD  supportability  statement 
(see  AR  75-15). 

4.  Environmental  statement  (see  AR 
200-1  and  32  CFR  651). 

5.  Statement  of  airworthiness  qualifica¬ 
tion  (see  AR  70-62). 

6.  SSRA  for  residual  hazards 
(see  MIL-STD-882E). 

7.  ATEC  (Army  Evaluation  Center) 
safety  confirmation  (see  AR  385-10). 

8.  Surface  or  weapon  danger  zone 
(see  AR  385-63). 

9.  Final  hazard  classification  (see  49 
CFR  173  and  TB  700-2). 

10  NRC  license  (see  10  CFR  chap  1). 

1 1 .  Army  Fuze  Safety  Review  Board 
Certification  (see  AR  385-10). 

12.  Energetic  materials  qualification 
(see  local  policy). 

13.  Ignition  System  Safety  Review 
Board  Certification  (see 

MIL— STD— 1 901  and  STANAG  4368. 

14.  Safety  review  of  TMs  (see  AR 
25-30). 

15.  Results  of  safety  inspections  and 
analyses. 

16.  Software  safety  statement. 

-System  safety  aspects  have  been  re¬ 
viewed  and  verified  by  the  supporting 
safety  office. 

-All  known  safety  hazards  have  been 
eliminated  or  accepted  through  the 
SSFtA  process  in  accordance  with  AR 
385-10. 

-All  statutory  requirements  are  met. 
-Applicable  regulatory  requirements 
are  met. 

LCMC  Safety  Office 

Suitability 

17.  TRADOC  training  assessment 
(statement  of  adequacy  of  institutional 
training  support)  (see  AR  350-1). 

-Training  is  determined  adequate  per 
AR  350-1. 

Force  modernization  propo¬ 
nent 

Supportable 

Support  strategy  to 
meet  Soldier’s  require¬ 
ments 

18.  Supportability  certification — will 
also  address  support  materiel  (COEI 
and  ASIOE),  end  item, 

and  software  (see  AR  700-127). 

19.  USATA  supportability  statement  on 
TMDE  or  ATE  (see  AR  750-43). 

20.  SDDC  TEA  transportability  state¬ 
ment  (see  AR  70-47).2 

21 .  Software  supportability  statement 
(normally  provided  by  the  LCMC  SEC). 

-Key  LCSP  performance  aspects  have 
been  achieved  as  determined  by  the 
functional  authorities. 

-Support  equipment  is  identified  and 
documented  at  the  appropriate  organi¬ 
zation;  TMDE  supportability  has  been 
addressed;  footprint  is  minimized. 
Transportability  has  been  evaluated  by 
SDDC  and  documented  accordingly. 

Lead  LCMC  ILS  center  or  ILS 
directorate 

CAPDEV  acceptance 

22.  CAPDEV  acceptance  of  materiel 
for  issues  or  restrictions 

-CAPDEV  acceptance  or  non-accept¬ 
ance  of  the  materiel  planned  for  a  TRM 
signed  by  a  general  officer  or  civilian 
equivalent 

CAPDEV 

Legend  for  Table  4-8: 

TRADOC  may  add  additional  requirements  from  table  4-2  and  table  4-3  not  already  required,  or  may  tailor  required  activities  based  upon  the  scope  and 
use  of  training  materiel. 


Notes: 

1  A  HHA  is  required  only  when  materiel  is  fielded  to  TRADOC  schools  and  training  sites.  Coordinate  with  the  U.S.  Army  Public  Health  Command  (Health 
Hazard  Assessment  Program)  for  inclusion  of  potential  health  hazard  information  in  course  curriculum  or  in  user  manuals. 

2  The  SDDC  transportability  statement  is  not  required  if  a  system  is  found  to  be  a  transportability  nonproblem  item  in  accordance  with  AR  70-47. 
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